------------------------------------------------------------
revno: 3035.4.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 39897-6.0
timestamp: Thu 2009-01-15 12:17:57 -0200
message:
Bug#39897: lock_multi fails in pushbuild: timeout waiting for processlist
The problem is that relying on the "Table lock" thread state in
its current position to detect that a thread is waiting on a lock
is race prone. The "Table lock" state change happens before the
thread actually tries to grab a lock on a table.
The solution is to move the "Table lock" state so that its set
only when a thread is actually going to wait for a lock. The state
change happens after the thread fails to grab the lock (because it
is owned by other thread) and proceeds to wait on a condition.
This is considered part of work related to WL#4284 "Transactional
DDL locking"
Warning: this patch contains an incompatible change.
When waiting on a lock in thr_lock.c, the server used to display "Locked"
processlist state. After this patch, the state is "Table lock".
The new state was actually intended to be display since year 2002,
when Monty added it. But up until removal of thd->locked boolean
member, this state was ignored by SHOW PROCESSLIST code.
mysql-test/r/lock_multi.result:
A style fix.
mysql-test/r/sp-threads.result:
Changed output of SHOW PROCESSLIST (new wait state).
mysql-test/t/lock_multi.test:
Use a more accurate state description when waiting inside thr_lock.c.
mysql-test/t/lock_sync.test:
Use a more accurate state description when waiting inside thr_lock.c.
mysql-test/t/multi_update.test:
Use a more accurate state description when waiting inside thr_lock.c.
mysql-test/t/query_cache_28249.test:
Use a more accurate state description when waiting inside thr_lock.c.
mysql-test/t/sp_notembedded.test:
Use a more accurate state description when waiting inside thr_lock.c.
mysql-test/t/status.test:
Use a more accurate state description when waiting inside thr_lock.c.
mysys/thr_lock.c:
Update thread state while waiting for a table lock.
sql/lock.cc:
State change was moved inside thr_lock.c.
----------------------------------------------------------
revno: 2630.4.38
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-4144
timestamp: Wed 2008-06-25 22:07:06 +0400
message:
WL#4144 - Lock MERGE engine children.
Committing a version of the patch merged with WL#3726
on behalf of Ingo.
Step #1: Move locking from parent to children.
MERGE children are now left in the query list of tables
after inserted there in open_tables(). So they are locked
by lock_tables() as all other tables are.
The MERGE parent does not store locks any more. It appears
in a MYSQL_LOCK with zero lock data. This is kind of a "dummy"
lock.
All other lock handling is also done directly on the children.
To protect against parent or child modifications during LOCK
TABLES, the children are detached after every statement and
attached before every statement, even under LOCK TABLES.
The children table list is removed from the query list of tables
on every detach and on close of the parent.
Step #2: Move MERGE specific functionality from SQL layer
into table handler.
Functionality moved from SQL layer (mainly sql_base.cc)
to the table handler (ha_myisammrg.cc).
Unnecessary code is removed from the SQL layer.
Step #3: Moved all MERGE specific members from TABLE
to ha_myisammrg.
Moved members from TABLE to ha_myisammrg.
Renamed some mebers.
Fixed comments.
Step #4: Valgrind and coverage testing
Valgrind did not uncover new problems.
Added purecov comments.
Added a new test for DATA/INDEX DIRECTORY options.
Changed handling of ::reset() for non-attached children.
Fixed the merge-big test.
Step #5: Fixed crashes detected during review
Changed detection when to attach/detach.
Added new tests.
Backport also the fix for Bug#44040 "MySQL allows creating a
MERGE table upon VIEWs but crashes when using it"
include/my_base.h:
WL#4144 - Lock MERGE engine children
Added HA_EXTRA_ADD_CHILDREN_LIST and HA_EXTRA_IS_ATTACHED_CHILDREN
for MERGE table support
mysql-test/r/merge.result:
WL#4144 - Lock MERGE engine children
Fixed test result.
mysql-test/t/disabled.def:
Enable merge.test, which now is working again (WL#4144).
mysql-test/t/merge-big.test:
Fix the messages for wait_condition (merge with WL#3726).
mysql-test/t/merge.test:
WL#4144 - Lock MERGE engine children
Fixed one test to meet coding standards for tests
(upper case keywords, engine names as in SHOW ENGINES).
Fixed error codes.
Added a test for DATA/INDEX DIRECTORY.
mysys/thr_lock.c:
WL#4144 - Lock MERGE engine children
Added purecov comments.
sql/ha_partition.cc:
WL#4144 - Lock MERGE engine children
Added MERGE specific extra operations to ha_partition::extra().
Extended comments.
Changed function comment to doxygen style.
Fixed nomenclature: 'parameter' -> 'operation'.
sql/mysql_priv.h:
WL#4144 - Lock MERGE engine children
Removed declarations for removed functions.
sql/sql_base.cc:
WL#4144 - Lock MERGE engine children
Leave the children in the query list of tables after open_tables().
Set proper back links (prev_global).
Attach MERGE children before and detach them after every
statement. Even under LOCK TABLES.
Remove children from the query list when they are detached.
Remove lock forwarding from children to parent.
Moved MERGE specific functions to ha_myisammrg.cc.
Added purecov comments.
Backport the fix for Bug#44040 "MySQL allows creating a MERGE table upon VIEWs but crashes when using it"
sql/sql_table.cc:
WL#4144 - Lock MERGE engine children
Changed detection of MERGE tables.
sql/table.cc:
WL#4144 - Lock MERGE engine children
Moved is_children_attached() method from TABLE to ha_myisammrg.
sql/table.h:
WL#4144 - Lock MERGE engine children
Moved all MERGE specific members from TABLE to ha_myisammrg.
storage/myisammrg/ha_myisammrg.cc:
WL#4144 - Lock MERGE engine children
Set proper back links in the child list (prev_global).
Added a function for removal of the child list from the query list.
Remove children from the query list when the parent is closed.
Make parent lock handling a dummy (zero locks).
Moved MERGE specific functionality from SQL layer to here.
Moved all MERGE specific members from TABLE to ha_myisammrg.
Renamed children list pointers.
Added initialization and free for the children list mem_root.
Fixed comments.
Added purecov comments.
storage/myisammrg/ha_myisammrg.h:
WL#4144 - Lock MERGE engine children
Added method add_children_list().
Moved all MERGE specific members from TABLE to ha_myisammrg.
Renamed children list pointers.
Added a mem_root for the children list.
storage/myisammrg/myrg_extra.c:
WL#4144 - Lock MERGE engine children
Changed handling of ::reset() for non-attached children.
----------------------------------------------------------
revno: 2630.4.35
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-3726
timestamp: Wed 2008-06-25 16:44:00 +0400
message:
Fix a MyISAM-specific bug in the new implementation of
LOCK TABLES (WL#3726).
If more than one instance of a MyISAM table are open in the
same connection, all of them must share the same status_param.
Otherwise, unlock of a table may lead to lost records.
See also comments in thr_lock.c.
include/thr_lock.h:
Declare thr_lock_merge_status().
mysql-test/r/lock.result:
Update test results (WL#3726).
mysql-test/t/lock.test:
Add a test case for the situation when the same table is locked
twice by LOCK TABLES, and only one instance is updated.
mysys/thr_lock.c:
Move the code that makes sure all status_params of the same
table are shared into a separate function.
sql/lock.cc:
Make sure that status_param is shared when a table is reopened.
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/r/show_check.result
Text conflict in mysql-test/r/sp-code.result
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/t/disabled.def
Text conflict in mysql-test/t/show_check.test
Text conflict in mysys/my_delete.c
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/log.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/repl_failsafe.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_parse.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_yacc.yy
Text conflict in storage/myisam/ha_myisam.cc
Corrected results for
stm_auto_increment_bug33029.reject 2009-12-01
20:01:49.000000000 +0300
<andrei> @@ -42,9 +42,6 @@
<andrei> RETURN i;
<andrei> END//
<andrei> CALL p1();
<andrei> -Warnings:
<andrei> -Note 1592 Statement may not be safe to log in statement
format.
<andrei> -Note 1592 Statement may not be safe to log in statement
format.
There should be indeed no Note present because there is in fact autoincrement
top-level query in sp() that triggers inserting in yet another auto-inc table.
(todo: alert DaoGang to improve the test).
NOTE: backporting BUG#42150 into next-mr
Includes latest Andrei's patch (see [2 Feb 18:40] Bugs System)
and merge.test post-push fix (see [3 Feb 18:04] Bugs System)
The reason of the bug appeared to be overreacting on absense of a
binlog file although the file name had been presented in in the master
binlog index file.
By convention, there should have been only a warning printed and the rest of
`reset master' logics completed. This did not happen on windows
due to incorrect value of my_errno returned from nt_share_delete().
Fixed with correcting my_errno assignment in nt_share_delete() to be ENOENT in
he event of no binlog file. Some minor refactoring has been made.
-------------------------------------------------------------
revno: 2877
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 35164-6.0
timestamp: Wed 2008-10-15 19:53:18 -0300
message:
Bug#35164: Large number of invalid pthread_attr_setschedparam calls
Bug#37536: Thread scheduling causes performance degradation at low thread count
Bug#12702: Long queries take 100% of CPU and freeze other applications under Windows
The problem is that although having threads with different priorities
yields marginal improvements [1] in some platforms [2], relying on some
statically defined priorities (QUERY_PRIOR and WAIT_PRIOR) to play well
(or to work at all) with different scheduling practices and disciplines
is, at best, a shot in the dark as the meaning of priority values may
change depending on the scheduling policy set for the process.
Another problem is that increasing priorities can hurt other concurrent
(running on the same hardware) applications (such as AMP) by causing
starvation problems as MySQL threads will successively preempt lower
priority processes. This can be evidenced by Bug#12702.
The solution is to not change the threads priorities and rely on the
system scheduler to perform its job. This also enables a system admin
to increase or decrease the scheduling priority of the MySQL process,
if intended.
Furthermore, the internal wrappers and code for changing the priority
of threads is being removed as they are now unused and ancient.
1. Due to unintentional side effects. On Solaris this could artificially
help benchmarks as calling the priority changing syscall millions of
times is more beneficial than the actual setting of the priority.
2. Where it actually works. It has never worked on Linux as the default
scheduling policy SCHED_OTHER only accepts the static priority 0.
configure.in:
Remove checks for functions that are not used anymore.
include/config-netware.h:
Remove unused define.
include/my_pthread.h:
Remove thread priority changing wrappers.
mysys/my_pthread.c:
Remove thread priority changing wrappers. They do not work properly
and their implementations were incorrectly protected by a check for
HAVE_PTHREAD_SETSCHEDPARAM.
mysys/thr_alarm.c:
Remove meaningless (100) increase of a thread priority.
sql/mysql_priv.h:
Remove meaningless thread priority values.
sql/mysqld.cc:
Don't change thread priorities.
sql/slave.cc:
Don't change thread priorities.
sql/slave.h:
Update function prototype.
sql/sql_parse.cc:
Don't change thread priorities.
sql/sql_prepare.cc:
Don't change thread priorities.
sql/unireg.h:
Mark flag as obsolete.
storage/innobase/handler/ha_innodb.cc:
Remove use of obsolete flag and associated behavior.
storage/innobase/include/srv0srv.h:
Remove use of obsolete flag and associated variables.
storage/innobase/os/os0thread.c:
Remove use of obsolete flag and associated behavior.
storage/innobase/srv/srv0srv.c:
Remove use of obsolete flag and associated variables.
Testing for presence of stuff in a hash inside the function
that's filling in the hash creates chicken-and-egg type of problems.
This results in test suite failures in mysql-pe in debug mode and
adds bad initialization dependency in 5.1.
Fixed by removing the debug code.
Not all my_hash_insert() calls are checked for return value.
This patch adds appropriate checks and failure responses
where needed.
mysys/hash.c:
* Debug hook for testing failures in my_hash_insert()
------------------------------------------------------------
revno: 2597.4.17
revision-id: sp1r-davi@mysql.com/endora.local-20080328174753-24337
parent: sp1r-anozdrin/alik@quad.opbmk-20080328140038-16479
committer: davi@mysql.com/endora.local
timestamp: Fri 2008-03-28 14:47:53 -0300
message:
Bug#15192 "fatal errors" are caught by handlers in stored procedures
The problem is that fatal errors (e.g.: out of memory) were being
caught by stored procedure exception handlers which could cause
the execution to not be stopped due to a continue handler.
The solution is to not call any exception handler if the error is
fatal and send the fatal error to the client.
mysql-test/r/sp-error.result:
Add test case result for Bug#15192
mysql-test/t/sp-error.test:
Add test case for Bug#15192
mysys/my_alloc.c:
Pass flag to signal fatal error in memory root allocations.
sql/event_data_objects.cc:
Use init_sql_alloc to initialize memory roots, which uses
the sql error handler to push errors.
sql/ha_partition.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/item_func.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/item_subselect.cc:
Remove redundant fatal error, memory root already pushes error.
sql/opt_sum.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/sp_head.cc:
Allocator already sets fatal error.
sql/sql_class.h:
A error must exist for it to be fatal. Pass flag to signal fatal
error instead of calling fatal_error.
sql/sql_insert.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/sql_list.h:
Pass flag to signal fatal error instead of calling fatal_error.
sql/sql_parse.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/sql_partition.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/sql_select.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/sql_servers.cc:
Use init_sql_alloc to initialize memory roots, which uses
the sql error handler to push errors.
sql/sql_show.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/sql_trigger.cc:
Use init_sql_alloc to initialize memory roots, which uses
the sql error handler to push errors.
sql/sql_update.cc:
Pass flag to signal fatal error instead of calling fatal_error.
sql/tztime.cc:
Use init_sql_alloc to initialize memory roots, which uses
the sql error handler to push errors.
BUILD-CMAKE:
WL#5161 : Documentation on how to build with CMake on Unix/Windows
BUILD/Makefile.am:
Add new file
BUILD/autorun.sh:
WL#5161 : use choose_configure instead of autotools configure script
(choose configure will call cmake if cmake is available)
BUILD/choose_configure.sh:
WL#5161 : use choose_configure instead of autotools configure script
(choose configure will call cmake if cmake is available)
CMakeLists.txt:
WL#5161 : Rewrite top-level CMakeLists.txt.
Remove Windows specifics
- compiler flags handling moved to configure.cmake
- storage engine/plugin stuff moved into cmake/plugin.cmake
- copy docs
Makefile.am:
Added new files
client/CMakeLists.txt:
WL#5161 : Rewrite CMakeLists.txt to be platform-independent
Handle packagng (add INSTALL commands)
cmake/Makefile.am:
WL#5161 : use choose_configure instead of autotools configure script
(choose configure will call cmake if cmake is available)
cmake/abi_check.cmake:
Custom targets for abi_check (for cmake)
cmake/bison.cmake:
- Check bison availability
- Add RUN_BISON macro (used to create sql_yacc.cc and sql_yacc.h)
cmake/cat.cmake:
Add helper script to concatenate files.
cmake/character_sets.cmake:
Handle configuration parameters WITH_EXTRA_CHARSETS
cmake/check_minimal_version.cmake:
Helper script to check the minimum required version of cmake
cmake/configure.pl:
Add perl script to convert ./configure parameters for cmake
cmake/create_initial_db.cmake.in:
Add script helper to create initial database.
(on Windows, we pack initial db with the redistribution
package)
cmake/do_abi_check.cmake:
Perform abi check
cmake/dtrace.cmake:
Handle dtrace in CMake Build.
Check for dtrace availablility,
run dtrace -G on solaris in prelink step
cmake/dtrace_prelink.cmake:
Run dtrace -G on Solaris in pre-link step,
link the object it creates together with library or
executable
cmake/install_macros.cmake:
Helper macros for packaging
(install pdb on Windows, install symlinks on Unix)
cmake/make_dist.cmake.in:
"make dist" -
- pack autotools ./configure script with the source
(renamed to configure.am)
- pack bison output
cmake/merge_archives_unix.cmake.in:
script to merge static libraries on Unix
cmake/misc.cmake:
Build helper macros
- MERGE_STATIC_LIBS
We use it when building client library and embedded
(avoid recompilation)
- Convert source file paths to absolute names.
We use it in to locate files of a different project,
when the files need to be recompiled (e.g in embedded
several storage engines are recompiled with
-DEMBEDDED_LIBRARY)
cmake/mysql_version.cmake:
Extract version info from configure.in
Handle package names.
cmake/plugin.cmake:
Rewrote storage/mysql_storage_engine.cmake to handle
other types of plugins and do it in OS-independent manner.
cmake/readline.cmake:
Macros to handle WITH_READLINE/WITH_LIBEDIT parameters
cmake/ssl.cmake:
Add macros to handle WITH_SSL parameter.
cmake/stack_direction.c:
Helper to check stack direction.
cmake/zlib.cmake:
Add macros to handle WITH_ZLIB parameter
cmd-line-utils/libedit/CMakeLists.txt:
Build libedit with cmake.
cmd-line-utils/libedit/Makefile.am:
Add new file
cmd-line-utils/readline/CMakeLists.txt:
Build readline with CMake.
cmd-line-utils/readline/Makefile.am:
Add new file
config.h.cmake:
WL#5161 : Add config.h template for cmake
configure.cmake:
WL#5161 : Add platform tests ( for cmake)
configure.in:
Added new subdirectories
dbug/CMakeLists.txt:
WL#5161
extra/CMakeLists.txt:
WL#5161
extra/yassl/CMakeLists.txt:
WL#5161
extra/yassl/taocrypt/CMakeLists.txt:
WL#5161
include/Makefile.am:
Add new file
include/keycache.h:
remove configure-win.h and remove HUGE_PTR defined there.
include/my_global.h:
use my_config.h for Windows, not config-win.h anymore
include/my_pthread.h:
- Move thread_safe_increment from config-win.h to other headers
(config-win.h is not used anymore)
- Declare pthread_cancel on Windows (it is used in daemon_example)
include/my_sys.h:
Add malloc.h on Windows (we use -D_WIN32_LEAN_AND_MEAN now, and
with this define malloc.h is not included automatically via windows.h)
include/mysql/plugin.h:
Handle pure-C plugins with Microsoft compiler.
include/thr_alarm.h:
remove rf_SetTimer that used to be defined in config-win.h
Replace with UINT_PTR (we do not use config-win.h anymore
and typedef was needed in this single place only)
libmysql/CMakeLists.txt:
Avoid pointless recompilation of source files
in client library if possible. Merge static
libs (dbug, mysys) to create static client
library.
libmysqld/CMakeLists.txt:
Avoid pointless recompilation of source files
when building embedded. Instead, merge dbug and
mysys (and some other static libs) into embedded.
libmysqld/examples/CMakeLists.txt:
Embedded compilation on Unix
libmysqld/lib_sql.cc:
Do not define THD::clear_error() in lib_sql.cc
for embedded. Instead, use the same inline
definition from sql_class.h as in none-embedded
case (fixes duplicate symbol errors on Windows
and removes pointless #ifdef EMBEDDED)
man/CMakeLists.txt:
Install man files.
man/Makefile.am:
Add new file.
mysql-test/CMakeLists.txt:
Install mysql-test files
mysql-test/Makefile.am:
Add new files
mysql-test/lib/My/ConfigFactory.pm:
Allow testing with mtr in out-of-source builds.
mysql-test/lib/My/Find.pm:
the build configurations are now also available on Unix
Xcode on Mac uses the Release, RelwithDebinfo and Debug
subdirectories for executables. Earlier, build configurations
were available only on Windows.
mysql-test/lib/My/SafeProcess.pm:
Allow testing with mtr in out-of-source builds.
mysql-test/lib/My/SafeProcess/CMakeLists.txt:
Port CMakeLists.txt to Unix
mysql-test/lib/My/SafeProcess/safe_kill_win.cc:
add stdlib.h (to be able to compile with -DWIN32_LEAN_AND_MEAN)
mysql-test/lib/My/SafeProcess/safe_process_win.cc:
Add stdlib.h (to be able to compile with -DWIN32_LEAN_AND_MEAN)
define JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE if not defined
(can happen using MinGW compiler that comes with old SDK)
mysql-test/mtr.out-of-source:
Allow testing with mtr in out-of-source builds.
mysql-test/mysql-test-run.pl:
Allow testing with mtr in out-of-source builds.
Use common find_plugin macro for Windows and unix.
mysql-test/t/fulltext_plugin.test:
This test can now run on Windows as well.
mysys/CMakeLists.txt:
Port CMakeLists.txt to Unix
mysys/my_create.c:
config-win.h is dead => NO_OPEN_3 is never defined.
mysys/my_getsystime.c:
config-win.h is dead => define OFFSET_TO_EPOCH where it is used.
mysys/my_winthread.c:
Add win32 pthread_cancel - used by daemon_example
mysys/mysys_priv.h:
config-win.h is dead => include <sys/stat.h> where it is used
fix prototype of my_win_(f)stat
plugin/daemon_example/CMakeLists.txt:
Compile daemon_example with CMake
plugin/daemon_example/Makefile.am:
Add new file
plugin/fulltext/CMakeLists.txt:
Compile full-text example with CMake
plugin/fulltext/Makefile.am:
Add new file.
plugin/semisync/CMakeLists.txt:
Fix semisync to use common macro for plugins.
regex/CMakeLists.txt:
Use absolute filenames, when adding regex library
(we recompile files in embedded, and want to locate
sources via GET_TARGET_PROPERTY( ... SOURCES ..))
regex/regex2.h:
Remove pointless typedef (produces error with MinGW compiler)
scripts/CMakeLists.txt:
Add configure/install for scripts
sql-bench/CMakeLists.txt:
install sql-bench files
sql-bench/Makefile.am:
Add new file
sql/CMakeLists.txt:
Port CmakeLists.txt to Unix
sql/nt_servc.cc:
compile server with -DWIN32_LEAN_AND_MEAN
sql/share/CMakeLists.txt:
Install charsets
sql/share/Makefile.am:
Add new file
sql/sql_builtin.cc.in:
Handle pure-C plugins on Windows.
sql/sql_class.h:
Use the same clear_error macro in embedded and not embedded.
Fixes pointless #ifdef and avoids duplicate symbols when linking
on Windows.
storage/Makefile.am:
storage/mysql_storage_engine.cmake => cmake/plugin.cmake
storage/archive/CMakeLists.txt:
Add names for static and dynamic plugin libraries.
Link archive with zlib
storage/blackhole/CMakeLists.txt:
Add names for static and dynamic storage
engine libraries
storage/csv/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/example/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/federated/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/heap/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/ibmdb2i/CMakeLists.txt:
Better port for ibmdb2i plugin
storage/innobase/CMakeLists.txt:
Run system checks.
Add names for static and dynamic storage engine
libraries.
storage/innobase/include/page0page.ic:
Fix compile error on OpenSolaris.
storage/myisam/CMakeLists.txt:
Port CmakeLists.txt to Unix
storage/myisammrg/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/mysql_storage_engine.cmake:
storage/mysql_storage_engine.cmake => cmake/plugin.cmake
support-files/CMakeLists.txt:
Configure and install some files from support-files.
support-files/Makefile.am:
Add new file
tests/CMakeLists.txt:
In general case, mysqlclient library can be dependent
on C++ runtime(if it includes yassl and is not compiled
with gcc or MSVC)
unittest/mysys/CMakeLists.txt:
Add unit tests
unittest/mysys/Makefile.am:
Add new file
unittest/mytap/CMakeLists.txt:
Add library for unit tests
unittest/mytap/Makefile.am:
Add new file
unittest/mytap/tap.c:
fix function definitions to match declarations
win/create_def_file.js:
Fix link error with intel compiler (icl
defines of special label for exception handler)
The presence of "--skip" parameters is obscure, when it should be
obvious from the text.
Now, for boolean options, when they're default to ON and the --skip
is more useful parameter, then tell the user of its existence.
Backported from 6.0-codebase, revid 2572.14.1
"What do you mean, there's a bug? There isn't even code!"
There was some token code for plug-in variables of the SET type,
but clearly this never worked, or was subject to massive bit rot
since. Bug-fixes ... fail-safes ... tests -- fais au mieux, mon chou!
mysys/my_getopt.c:
SETs set-up should set up a default value, but no min/max bounding.
mysys/typelib.c:
fail-safe requested by serg: don't try to skip separator when we're
already at end of string.
sql/sql_plugin.cc:
check_func_set:
Initialize error_len as find_set() will only update it on error,
and we're using the value to see whether an error has occurred (!= 0),
so we'd better not have a random val in there.
value_ptr:
There's no guarantee we're handed string lengths, so play it safe!
Use prepared string lengths where possible for minimum speed gain,
otherwise determine on the fly!
The prefix --skip- didn't work on 64 bit big endian machines
because of how the value pointer was casted.
mysys/my_getopt.c:
* Use the interface! The value pointer must correspond to the type mask or it will break on big endian platforms.
Concurrent execution of statements which require non-table-level
write locks on several instances of the same table (such as
SELECT ... FOR UPDATE which uses same InnoDB table twice or a DML
statement which invokes trigger which tries to update same InnoDB
table directly and through stored function) and statements which
required table-level locks on this table (e.g. LOCK TABLE ... WRITE,
ALTER TABLE, ...) might have resulted in a deadlock.
The problem occured when a thread tried to acquire write lock
(TL_WRITE_ALLOW_WRITE) on the table but had to wait since there was
a pending write lock (TL_WRITE, TL_WRITE_ALLOW_READ) on this table
and we failed to detect that this thread already had another instance
of write lock on it (so in fact we were trying to acquire recursive
lock) because there was also another thread holding write lock on the
table (also TL_WRITE_ALLOW_WRITE). When the latter thread released
its lock neither the first thread nor the thread trying to acquire
TL_WRITE/TL_WRITE_ALLOW_READ were woken up (as table was still write
locked by the first thread) so we ended up with a deadlock.
This patch solves this problem by ensuring that thread which
already has write lock on the table won't wait when it tries
to acquire second write lock on the same table.
mysql-test/r/lock_sync.result:
Added test case for bug #45143 "All connections hang on concurrent
ALTER TABLE".
mysql-test/t/lock_sync.test:
Added test case for bug #45143 "All connections hang on concurrent
ALTER TABLE".
mysys/thr_lock.c:
Ensured that thread can acquire write lock on the table without
waiting if it already has write lock on it even if there are other
threads holding write locks on this table (this is normal situation
for, e.g., TL_WRITE_ALLOW_WRITE type of lock).
Adjusted comments to better explain why it is OK to do so and added
asserts to prevent introduction of scenarios in which this can cause
problems.
Conflicts
=========
Text conflict in .bzr-mysql/default.conf
Text conflict in libmysqld/CMakeLists.txt
Text conflict in libmysqld/Makefile.am
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/extra/rpl_tests/rpl_row_sp006.test
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_create_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_sp006.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/Makefile.am
Text conflict in sql/log_event_old.cc
Text conflict in sql/rpl_rli.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_binlog.cc
Text conflict in sql/sql_lex.h
21 conflicts encountered.
NOTE
====
mysql-5.1-rpl-merge has been made a mirror of mysql-next-mr:
- "mysql-5.1-rpl-merge$ bzr pull ../mysql-next-mr"
This is the first cset (merge/...) committed after pulling
from mysql-next-mr.
added code to exit a tool if the forced config file wasn't found
per-file comments:
client/mysql.cc
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysql_upgrade.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqladmin.cc
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlcheck.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqldump.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlimport.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlshow.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlslap.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
mysql-test/t/mysql.test
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
test added
sql/mysqld.cc
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
storage/myisam/myisamchk.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
storage/myisam/myisampack.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
The bug is a compilation issue:
Function "find_key_block()" had thread operations
which were not guarded by "#if THREAD", add that now.
mysys/mf_keycache.c:
Without thread-support, we are in a single-threaded world,
so there is no parallelism, and the keycache cannot be
within a resize operation while this request is being processed.
The pthread_cond_wait implementations for windows might
dead lock in some rare circumstances.
1) One thread (I) enter a timed wait and at a point in
time ends up after mutex unlock and before
WaitForMultipleObjects(...)
2) Another thread (II) enters pthread_cond_broadcast.
Grabs the mutex and discovers one waiter. It set
the broadcast event and closes the broadcast gate
then unlocks the mutex.
3) A third thread (III) issues a pthread_cond_signal.
It grabs the mutex, discovers one waiter, sets the
signal event then unlock the mutex.
4) The first threads (I) enters WaitForMultipleObjects
and finds out that the signal object is in a
signalled state and exits the wait.
5) Thread (I) grabs the mutex and checks result status.
The number of waiters is decreased and becomes equal
to 0. The event returned was a signal event so the
broadcast gate isn't opened. The mutex is released.
6) Thread (II) issues a new broadcast. The mutex is
acquired but the number of waiters are 0 hence
the broadcast gate remains closed.
7) Thread (I) enters the wait again but is blocked by
the broadcast gate.
This fix resolves the above issue by always resetting
broadcast gate when there are no more waiters in th queue.
mysys/my_wincond.c:
* Always reset the broadcast gate if there are no more waiters left.
Options loaded from config files were added before command line
arguments, and they were parsed together, which could interprete
the following:
option-a
option-b
as --option-a=--option-b if 'option-a' requires a value, and
caused confusing.
Because all options that requires a value are always given in
the form '--option=value', so it's an error if there is no
'=value' part for such an option read from config file.
This patch added a separator to separate the arguments from
config files and that from command line, so that they can be
handled differently. And report an error for options loaded
from config files that requires a value and is not given in the
form '--option=value'.
http://lists.mysql.com/commits/59686
Cleanup pthread_self(), pthread_create(), pthread_join() implementation on Windows.
Prior implementation is was unnecessarily complicated and even differs in embedded
and non-embedded case.
Improvements in this patch:
* pthread_t is now the unique thread ID, instead of HANDLE returned by beginthread
This simplifies pthread_self() to be just straight GetCurrentThreadId().
prior it was much art involved in passing the beginthread() handle from the caller
to the TLS structure in the child thread ( did not work for the main thread of
course)
* remove MySQL specific my_thread_init()/my_thread_end() from pthread_create.
No automagic is done on Unix on pthread_create(). Having the same on Windows will
improve portability and avoid extra #ifdef's
* remove redefinition of getpid() - it was defined as GetCurrentThreadId()
2677 Vladislav Vaintroub 2008-11-04
CMakeLists.txt files cleanup
- remove SAFEMALLOC and SAFE_MUTEX definitions that were
present in *each* CMakeLists.txt. Instead, put them into top level
CMakeLists.txt, but disable on Windows, because
a) SAFEMALLOC does not add any functionality that is not already
present in Debug C runtime ( and 2 safe malloc one on top of the other
only unnecessarily slows down the server)
b)SAFE_MUTEX does not work on Windows and have been
explicitely disabled on Windows with #undef previously. Fortunately,
ntdll does pretty good job identifying l problems with
CRITICAL_SECTIONs.
DebugBreak()s on using uninited critical section, unlocking unowned
critical section)
-Also, remove occationally used -D_DEBUG (added by compiler
anyway)
sql/udf_example.c:
use unixish end of line
This patch provides performance improvements:
- send_fields() when character_set_results = latin1
is now about twice faster for column/table/database
names, consisting on ASCII characters.
Changes:
- Protocol doesn't use "convert" temporary buffer anymore,
and converts strings directly to "packet".
- General conversion optimization: quick conversion
of ASCII strings was added.
modified files:
include/m_ctype.h
- Adding a new flag.
- Adding a new function prototype
libmysqld/lib_sql.cc
- Adding quick conversion method for embedded library:
conversion is now done directly to result buffer,
without using a temporary buffer.
mysys/charset.c
- Mark all dynamic ucs2 character sets as non-ASCII
- Mark some dymamic 7bit and 8bit charsets as non-ASCII
(for example swe7 is not fully ASCII compatible).
sql/protocol.cc
- Adding quick method to convert a string directly
into protocol buffer, without using a temporary buffer.
sql/protocol.h
- Adding a new method prototype
sql/sql_string.cc
Optimization for conversion between two ASCII-compatible charsets:
- quickly convert ASCII strings,
switch to mc_wc->wc_mb method only when a non-ASCII character is met.
- copy four ASCII characters at once on i386
strings/conf_to_src.c
- Marking non-ASCII character sets with a flag.
strings/ctype-extra.c
- Regenerating ctype-extra.c by running "conf_to_src".
strings/ctype-uca.c
- Marking UCS2 character set as non-ASCII.
strings/ctype-ucs2.c
- Marking UCS2 character set as non-ASCII.
strings/ctype.c
- A new function to detect if a 7bit or 8bit character set
is ascii compatible.
http://lists.mysql.com/commits/57725
Vladislav Vaintroub 2008-11-03
Cleanup CMakeLists.txt(s) - remove winsock2 (ws2_32) from
TARGET_LINK_LIBRARIES.
Every exe or dll linked with mysys needs ws2_32, because
mysys uses winsock function WSAStartup in my_init().
However, there is no need to explicitely add ws2_32 to
the list of TARGET_LINK_LIBRARIES multiple times.
Visual Studio comes with a handy pragma that tells linker
to add library. So patch replaces bunch of ws2_32 in
CMakeLists with single pragma comment(lib,"ws2_32")
in my_init.c
Additionally, reference to non-existing "debug" library
has been removed from TARGET_LINK_LIBRARIES. The correct
name of the library is "dbug".
htttp://lists.mysql.com/commits/50957?f=plain
Always use TLS functions instead of __declspec(thread) to access
thread local storage variables.
The change removes the necessity to recomplile the same source
files twice - with USE_TLS for DLLs and without USE_TLS for EXEs.
Real benefit of this change is better readability and maintainability
of TLS functions within MySQL.
There is a performance loss using TlsXXX functions compared to __declspec
but the difference is negligible in practice. In a sysbench-like benchmark
I ran with with TlsGetValue, pthread_[get|set]_specific was called 600000000
times and took 0.17sec of total 35min CPU time, or 0.008%.
Backport from 6.0 to 5.1.
Only those sync points are included, which are used in debug_sync.test.
The Debug Sync Facility allows to place synchronization points
in the code:
open_tables(...)
DEBUG_SYNC(thd, "after_open_tables");
lock_tables(...)
When activated, a sync point can
- Send a signal and/or
- Wait for a signal
Nomenclature:
- signal: A value of a global variable that persists
until overwritten by a new signal. The global
variable can also be seen as a "signal post"
or "flag mast". Then the signal is what is
attached to the "signal post" or "flag mast".
- send a signal: Assign the value (the signal) to the global
variable ("set a flag") and broadcast a
global condition to wake those waiting for
a signal.
- wait for a signal: Loop over waiting for the global condition until
the global value matches the wait-for signal.
Please find more information in the top comment in debug_sync.cc
or in the worklog entry.
.bzrignore:
WL#4259 - Debug Sync Facility
Added the symbolic link libmysqld/debug_sync.cc.
CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added definition for ENABLED_DEBUG_SYNC.
configure.in:
WL#4259 - Debug Sync Facility
Added definition for ENABLED_DEBUG_SYNC.
include/my_sys.h:
WL#4259 - Debug Sync Facility
Added definition for the DEBUG_SYNC_C macro.
libmysqld/CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added sql/debug_sync.cc.
libmysqld/Makefile.am:
WL#4259 - Debug Sync Facility
Added sql/debug_sync.cc.
mysql-test/include/have_debug_sync.inc:
WL#4259 - Debug Sync Facility
New include file.
mysql-test/mysql-test-run.pl:
WL#4259 - Debug Sync Facility
Added option --debug_sync_timeout.
mysql-test/r/debug_sync.result:
WL#4259 - Debug Sync Facility
New test result.
mysql-test/r/have_debug_sync.require:
WL#4259 - Debug Sync Facility
New require file.
mysql-test/t/debug_sync.test:
WL#4259 - Debug Sync Facility
New test file.
mysys/my_static.c:
WL#4259 - Debug Sync Facility
Added definition for debug_sync_C_callback_ptr.
mysys/thr_lock.c:
WL#4259 - Debug Sync Facility
Added sync point "wait_for_lock".
sql/CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added debug_sync.cc and debug_sync.h.
sql/Makefile.am:
WL#4259 - Debug Sync Facility
Added debug_sync.cc and debug_sync.h.
sql/debug_sync.cc:
WL#4259 - Debug Sync Facility
New source file.
sql/debug_sync.h:
WL#4259 - Debug Sync Facility
New header file.
sql/mysqld.cc:
WL#4259 - Debug Sync Facility
Added opt_debug_sync_timeout.
Added calls to debug_sync_init() and debug_sync_end().
Fixed a purecov comment (unrelated).
sql/set_var.cc:
WL#4259 - Debug Sync Facility
Added server variable "debug_sync".
sql/set_var.h:
WL#4259 - Debug Sync Facility
Added declaration for server variable "debug_sync".
sql/share/errmsg.txt:
WL#4259 - Debug Sync Facility
Added error messages ER_DEBUG_SYNC_TIMEOUT and ER_DEBUG_SYNC_HIT_LIMIT.
sql/sql_base.cc:
WL#4259 - Debug Sync Facility
Added sync points "after_flush_unlock" and "before_lock_tables_takes_lock".
sql/sql_class.cc:
WL#4259 - Debug Sync Facility
Added initialization for debug_sync_control to THD::THD.
Added calls to debug_sync_init_thread() and debug_sync_end_thread().
sql/sql_class.h:
WL#4259 - Debug Sync Facility
Added element debug_sync_control to THD.
storage/myisam/myisamchk.c:
Fixed a typo in an error message string (unrelated).
- Create the "dummy" thread joinable and wait for it to
exit before continuing in 'my_thread_global_init'
- This way we know that the pthread library is initialized
by one thread only
Large pages allocator could not allocate more than 4 GB due to
incorrect size alignment.
mysys/my_largepage.c:
Large pages allocator could not allocate more than 4 GB due to
incorrect size alignment.
Option "--without-server" still not working in 5.1
The general approach is to make sure that source files
which require thread support are only compiled if the build
really needs thread support,
which means when the server is built or a thread-safe client
library.
This required several changes:
- Make sure the subdirectories "storage/" and "plugin/" are
only processed if the server is built, not ifclient-only.
- Make the compilation of some modules which inherently
require threading depend on thread supportin the build.
- Separate the handling of threading in "configure.in" from
that of server issues, threading is also needed in a
non-server build of a thread-safe client library.
Also, "libdbug" must get built even in a client-only build,
so "dbug/" must be in the list of client directories.
In addition, calls to thread functions in source files which
can be built without thread support must use the wrapper
functions which handle the non-threaded build.
So the modules "client/mysqlimport.c" and "client/mysqlslap.c"
must call "my_thread_end()" only via "mysql_thread_end()".
Makefile.am:
The directories "storage/" and "plugin/" contain files
which are needed for the server only, so their contents
is to be built only if a server is built.
They must not be named unconditionally, because building
their contents will fail unless threads are enabled.
These directories are now listed in the "configure"
variable "sql_server_dirs" which becomes part of
"sql_union_dirs" if the server is to be built.
client/mysqlimport.c:
Use the wrapper function "mysql_thread_end()" which
correctly handles the case of a non-threaded build.
client/mysqlslap.c:
Use the wrapper function "mysql_thread_end()" which
correctly handles the case of a non-threaded build.
configure.in:
Various changes to support builds "--without-server":
1) For the unit tests, we need "libdbug".
2) Separate the treatment of the server from that of the
thread-safe client library.
3) Introduce an "automake conditional" "NEED_THREAD"
which can be checked in some "Makefile.am".
4) Include "storage/" and "plugin/" in the list of
"sql_server_dirs" so that they are handled in the
top "Makefile.am" only if the server is to be built
(see the change in that file).
mysys/Makefile.am:
The code of "mf_keycache.c" in 5.1 is no longer safe
to be built without thread support.
(In 5.0, this was possible.)
Rather than fix these issues, which is tedious and risky,
avoid the need to ever build it without thread support:
It is needed in the server only, which needs thread support.
The only case where we build a "libmysys" without thread
support is for a non-threaded client, where "mf_keycache"
is not neded.
So its inclusion in the list of source files can depend
on the new conditional "NEED_THREAD".
unittest/mysys/Makefile.am:
Test program "my_atomic-t" is to verify the correct handling
of threads only, it cannot be built without thread support
and is not needed in such cases either.
Let its build depend on the new conditional "NEED_THREAD".
Bug#24509 - 2048 file descriptor limit on windows needs increasing, also
WL#3049 - improved Windows I/O
The patch replaces the use of the POSIX I/O interfaces in mysys on Windows with
the Win32 API calls (CreateFile, WriteFile, etc). The Windows HANDLE for the open
file is stored in the my_file_info struct, along with a flag for append mode
because the Windows API does not support opening files in append mode in all cases)
The default max open files has been increased to 16384 and can be increased further
by setting --max-open-files=<value> during the server start.
Another major change in this patch that almost all Windows specific file IO code
has been moved to a new file my_winfile.c, greatly reducing the amount of code
in #ifdef blocks within mysys, thus improving readability.
Minor enhancements:
- my_(f)stat() is changed to use __stati64 structure with 64 file size
and timestamps. It will return correct file size now (C runtime implementation
used to report outdated information)
- my_lock on Windows is prepared to handle additional timeout parameter
- after review : changed __WIN__ to _WIN32 in the new and changed code.
client/mysqlbinlog.cc:
fileno -> my_fileno
client/readline.cc:
fileno -> my_fileno
include/config-win.h:
Increase OS_FILE_LIMIT for Windows.
Remove O_SHARE - Windows does not support it. Its definition conflicts with
O_SHORT_LIVED, that has different semantics.
include/my_dir.h:
Use stat64 for stat() family of functions on Windows, because of 64 bit file size.
include/my_global.h:
Increased default value for open file limit to 16K
include/my_sys.h:
- my_file_info got new structure members - file handle and open flags
- 2 new Windows-only mysys functions : my_get_osfhandle and my_osmaperr,
modelled after Windows C runtime functions _get_osfhandle and _dosmaperr
libmysql/CMakeLists.txt:
new files my_winfile.c and my_winerr.c
mysql-test/suite/large_tests/r/lock_tables_big.result:
test for more then 2048 open file descriptors on Windows
mysql-test/suite/large_tests/t/lock_tables_big.test:
test for more then 2048 open file descriptors on Windows
mysys/CMakeLists.txt:
new files my_winfile.c and my_winerr.c
mysys/Makefile.am:
new files my_winfile.c and my_winerr.c
mysys/default_modify.c:
fileno -> my_fileno
mysys/my_chsize.c:
implementation of chsize on Windows now moved to my_winfile.c
mysys/my_create.c:
- my_sopen->my_win_open
- close open file before removing (won't generally work on Windows otherwise)
mysys/my_file.c:
On Windows, files returned by my_open will not start with 0, but 2048
(making it simple to detect incompatible mix of CRT and mysys io functions)
mysys/my_fopen.c:
fileno->my_win_fileno , fclose->my_win_fclose, fdopen->my_win_fdopen
Check for legal filename is done by my_win_[f]open functions
mysys/my_fstream.c:
fileno->my_fileno
mysys/my_lib.c:
Windows stat() functions are moved to my_winfile.c
mysys/my_lock.c:
Move Windows code under #ifdef to a separate function win_lock().
Add a parameter for lock wait timeout
mysys/my_mmap.c:
_get_osfhandle->my_get_osfhandle
mysys/my_open.c:
my_sopen->my_win_open (simpler interface)
mysys/my_pread.c:
moved most windows specific code to my_win_file.c
Use my_win_pread
mysys/my_quick.c:
Use my_win_read/my_win_write
mysys/my_read.c:
Moved most of windows specific code to my_win_file.c
Use my_win_read()
mysys/my_seek.c:
On Windows, use my_win_lseek() in my_seek()/my_tell()
Removed dead code (synchronization of lseeks)
Improved DBUG tracing (file position is ulonglong, not ulong)
mysys/my_static.c:
Removed array initialization. my_file_info_default is global variable
thus it is initialized with all zeros anyway
mysys/my_sync.c:
_commit->my_win_fsync
mysys/my_winerr.c:
New file my_winerr.c
Exports my_osmaperr modelled after undocumented C runtime
function _dosmaperr(). The problem with _dosmaperr() used previously is that
1) it is nowhere documented and thus code relying on it is not guaranteed to work
in subsequent releases on the C runtime
2) it is present only in static C runtime (mysqld does not link if compiled with
/MD)
mysys/my_winfile.c:
New file my_winfile.c
Implements ANSI/Posix file IO routines, when possible using native Windows IO, without
C runtime (C runtime dropped because of the 2048 file descriptor limit).
mysys/my_write.c:
write->my_win_write
mysys/mysys_priv.h:
Declaration of Windows Posix functions (private to mysys, shall not be visible
outside)
storage/innobase/handler/ha_innodb.cc:
mysys native Windows IO : correct innodb tmp file handling
mysql_tmpfile does not return valid CRT file descriptor, thus
it is not possible to dup() it. Instead, the native file handle has
to be duplicated and then converted to CRT descriptor.
storage/myisam/mi_locking.c:
_commit->my_sync
can crash under load
Merge from 5.0, after backport from 5.1/5.4 to 5.0.
This makes the fixes for
Bug 44068 (RESTORE can disable the MyISAM Key Cache)
Bug 40944 (Backup: crash after myisampack)
available to 5.1.
can crash under load
Backport from 5.1.
Does also include key cache fixes from:
Bug 44068 (RESTORE can disable the MyISAM Key Cache)
Bug 40944 (Backup: crash after myisampack)
include/keycache.h:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Added KEY_CACHE components in_resize and waiting_for_resize_cnt.
myisam/mi_preload.c:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Added code to allow LOAD INDEX to load indexes of different block size.
mysys/mf_keycache.c:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
.
Changed resize_key_cache() to not disable the key cache
after the flush phase. Changed queue handling to use
standard functions. Wake all threads waiting on resize_queue.
We can now have read/write threads waiting there (see below).
.
Combined add_to_queue() and the wait loops that were always
following it to the new function wait_on_queue().
Combined release_queue() and the condition that was always
preceding it to the new function release_whole_queue().
.
Added code to flag and respect the exceptional situation
BLOCK_IN_EVICTION.
.
Rewrote the resize branch of find_key_block().
.
Added code to the eviction handling in find_key_block()
to catch more exceptional cases.
.
Changed key_cache_read(), key_cache_insert() and key_cache_write()
so that they lock keycache->cache_lock whenever the key cache is
initialized. Checking for a disabled cache and incrementing and
decrementing the "resize counter" is always done within the lock.
Locking and unlocking as well as counting the "resize counter" is
now done once outside the loop. All three functions can now handle
a NULL return from find_key_block. This happens in the flush phase
of a resize and demands direct file I/O. Care is taken for
secondary requests (PAGE_WAIT_TO_BE_READ) to wait in any case.
Moved block status changes behind the copying of buffer data.
key_cache_insert() does now read the block if the caller did
supply less data than a full cache block.
key_cache_write() does now take care of parallel running flushes
(BLOCK_FOR_UPDATE, BLOCK_IN_FLUSHWRITE).
.
Changed free_block() to un-initialize block variables in the
correct order and respect an exceptional BLOCK_IN_EVICTION state.
.
Changed flushing to take care for parallel running writes.
Changed flushing to avoid freeing blocks in eviction.
Changed flushing to consider that parallel writes can move blocks
from the file_blocks hash to the changed_blocks hash.
Changed flushing to take care for other parallel flushes.
Changed flushing to assure that it ends with everything flushed.
Optimized normal flush at end of statement (FLUSH_KEEP),
but let other flush types be stringent.
.
Added some comments and debugging statements.
mysys/my_static.c:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Removed an unused global variable.
sql/ha_myisam.cc:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Moved an automatic (stack) variable to the scope where it is used.
sql/sql_table.cc:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Changed TL_READ to TL_READ_NO_INSERT in mysql_preload_keys.
CREATE TABLE...LIKE...
The mysql server option 'sync_frm' is ignored when table is created with
syntax CREATE TABLE .. LIKE..
Fixed by adding the MY_SYNC flag and calling my_sync() from my_copy() when
the flag is set.
In mysql_create_table(), when the 'sync_frm' is set, MY_SYNC flag is passed
to my_copy().
Note: TestCase is not attached and can be tested manually using debugger.
client/Makefile.am:
BUG#46591 - .frm file isn't sync'd with sync_frm enabled for
CREATE TABLE...LIKE...
add my_sync to sources as it is used in my_copy() method
include/my_sys.h:
BUG#46591 - .frm file isn't sync'd with sync_frm enabled for
CREATE TABLE...LIKE...
MY_SYNC flag is added to call my_sync() method
mysys/my_copy.c:
BUG#46591 - .frm file isn't sync'd with sync_frm enabled for
CREATE TABLE...LIKE...
my_sync() is method is called when MY_SYNC is set in my_copy()
sql/sql_table.cc:
BUG#46591 - .frm file isn't sync'd with sync_frm enabled for
CREATE TABLE...LIKE...
Fixed mysql_create_like_table() to call my_sync() when opt_sync_frm variable
is set
with gcc 4.3.2
This patch fixes a number of GCC warnings about variables used
before initialized. A new macro UNINIT_VAR() is introduced for
use in the variable declaration, and LINT_INIT() usage will be
gradually deprecated. (A workaround is used for g++, pending a
patch for a g++ bug.)
GCC warnings for unused results (attribute warn_unused_result)
for a number of system calls (present at least in later
Ubuntus, where the usual void cast trick doesn't work) are
also fixed.
client/mysqlmanager-pwgen.c:
A fix for warn_unused_result, adding fallback to use of
srand()/rand() if /dev/random cannot be used. Also actually
adds calls to rand() in the second branch so that it actually
creates a random password.
client/mysql.cc:
Remove leading whitespace.
Remove extra text after #else directive.
client/mysqldump.c:
Function does not take a parameter.
mysys/array.c:
buffer is a uchar pointer.
sql/item.cc:
Assert if it should not happen.
storage/myisam/mi_check.c:
Cast to expected type. This is probably a bug, but it is
casted in a similar way in another part of the code.
storage/ndb/include/mgmapi/ndb_logevent.h:
Apply fix from cluster team.
tests/mysql_client_test.c:
Remove extraneous slash.
decrease for INSERTs
Bulk inserts (multiple row, CREATE ... SELECT, INSERT ... SELECT) into
MyISAM tables were performed inefficiently. This was mainly affecting
use cases where read_buffer_size was considerably large (>256K) and low
number of rows was inserted (e.g. 30-100).
The problem was that during I/O cache initialization (this happens
before each bulk insert) allocated I/O buffer was unnecessarily
initialized to '\0'.
This was happening because of mess in flag values. MyISAM informs I/O
cache to wait for free space (if out of disk space) by passing
MY_WAIT_IF_FULL flag. Since MY_WAIT_IF_FULL and MY_ZEROFILL have the
same values, memory allocator was initializing memory to '\0'.
The performance gain provided with this patch may only be visible with
non-debug binaries, since safemalloc always initializes allocated memory
to 0xA5A5...
mysys/mf_iocache.c:
Remove MY_WAIT_IF_FULL from myflags before calling my_malloc
to prevent conflict with MY_ZEROFILL.
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin
bzr rm innobase # remove the builtin
Next step: build, test fixes.
- Define and pass compile time path variables as pre-processor definitions to
mimic the makefile build.
- Set new CMake version and policy requirements explicitly.
- Changed DATADIR to MYSQL_DATADIR to avoid conflicting definition in
Platform SDK header ObjIdl.h which also defines DATADIR.
This patch is a follow up to http://lists.mysql.com/commits/76678.
When an allocation failure occurs for the buffer in the dynamic
array, an error condition was being set. The dynamic array is
usable even if the memory allocation fails. Since in most cases
the thread can continue to work without any problems the error
condition should not be set here.
This patch adds logic to remove the error condition from being set
when the memory allocation for the buffer in dynamic array fails.
mysys/array.c:
Bug#43572 Handle failures from hash_init
Remove the MY_WME flag from the call to malloc in order to
prevent the error status from being set in the init_dynamic_array
method. Since this memory allocation failure is no longer
fatal this method has been modified to return FALSE
(indicate success) irrespective of array->buffer being
allocated.
Because of a regression introduced by bug#19027 the option --enable-foobar
doesn't work anymore for any plugin 'foobar'. The reason is that plugin
names are tristate options variables with optional parameters and integer
values are not accepted. Since the 'enable' prefix attempts to assign '1'
to the option the operation fails.
This patch translates any number n assigned to a plugin variable of type ENUM
to be the corresponding enumerated item. As a side effect --enable-foobar and
--disable-foobar will also start working again.
mysys/my_getopt.c:
* setval now accepts integer values for option variables of type ENUM.
The merge from http://lists.mysql.com/commits/76678 caused the
growth_size parameter to the my_init_dynamic_array function to
be ignored. This patch corrects the problem.
mysys/hash.c:
Bug#43572 Handle failures from hash_init
Replacing the last parameter to my_init_dynamic_array with
growth_size.
Failure to allocate memory for the hash->array element,
caused hash_init to return without initializing the other
members of the hash. Thus although the dynamic array
buffer may be allocated at a later point in the code, the
incompletely initialized hash caused fatal failures.
This patch moves the initialization of the other members
of the hash above the array allocation, so that the usage
of this hash will not result in fatal failures.
include/hash.h:
Bug#43572 Handle failures from hash_init
hash_inited is used to verify that the hash is
valid. After the change induced by the current
patch hash->array.buffer !=0 is not a valid check
for this condition, since, the dynamic array can
be allocated even at a later time. Bootstrap SQL
script is setting some variables, which are
actually not set due to this hash_inited issue.
Thus we get empty grant tables.
A better way to check if the hash is valid is
to verify that hash->blength is greater than 0.
mysys/hash.c:
Bug#43572 Handle failures from hash_init
Move the initialization of the other members
of the hash above the array allocation, so that
the usage of this hash will not result in fatal
failures.
Remove custom DLL loader code from innodb plugin code, use
symbols exported from mysqld.
storage/innodb_plugin/handler/ha_innodb.cc:
Remove a Win32 workaround for current_thd.
The original problem that innodb plugin used
value of TLS variable across DLL boundaries is
solved in MySQL server (current_thd is a function
not TLS variable now)
storage/innodb_plugin/handler/handler0alter.cc:
Remove custom delay loader
storage/innodb_plugin/handler/handler0vars.h:
Remove custom delay loader
storage/innodb_plugin/handler/i_s.cc:
Remove custom delay loader
storage/innodb_plugin/handler/win_delay_loader.cc:
Remove custom delay loader
storage/innodb_plugin/plug.in:
Remove commented out MYSQL_PLUGIN_STATIC,
CMake would not parse that correctly
It is not possible to prevent the server from starting if a mandatory
built-in plugin fails to start. This can in some cases lead to data
corruption when the old table name space suddenly is used by a different
storage engine.
A boolean command line option in the form of --foobar is automatically
created for every existing plugin "foobar". By changing this command line
option from a boolean to a tristate { OFF, ON, FORCE } it is possible to
specify the plugin loading policy for each plugin.
The behavior is specified as follows:
OFF = Disable the plugin and start the server
ON = Enable the plugin and start the server even if an error occurrs
during plugin initialization.
FORCE = Enable the plugin but don't start the server if an error occurrs
during plugin initialization.
mysql-test/lib/mtr_cases.pm:
* Changed --<pluginname> from a boolean to a tristate.
mysys/my_getopt.c:
* Changed --<pluginname> from boolean to tristate. Optional arguments
must still work for tristates. It is praxis that disable means value 0
and enable is value 1. Since plugin name is the only tristate with
optional arguments this principle will still hold.
sql/sql_plugin.cc:
* Changed --<pluginname> option from a boolean type to a tristate.
- FORCE will now terminate the server if the plugin fails to
initialize properly.
* Refactored prototypes for test_plugin_options() and construct_options()
to get rid of the 'enable' value pointer.
* Cleaned up code related to option name constructing.
* Added documentation
sql/sql_plugin.h:
* Introduced new member to st_plugin_int structure.
165 changesets with 23 conflicts:
Text conflict in mysql-test/r/lock_multi.result
Text conflict in mysql-test/t/lock_multi.test
Text conflict in mysql-test/t/mysqldump.test
Text conflict in sql/item_strfunc.cc
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/parse_file.cc
Text conflict in sql/slave.cc
Text conflict in sql/sp.cc
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.cc
Text conflict in sql/sql_crypt.cc
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_parse.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_view.cc
Text conflict in storage/innobase/handler/ha_innodb.cc
Text conflict in storage/myisam/mi_packrec.c
Text conflict in tests/mysql_client_test.c
Updates to Innobase, taken from main 5.1:
bzr: ERROR: Some change isn't sane:
File mysql-test/r/innodb-semi-consistent.result is owned by Innobase and should not be updated.
File mysql-test/t/innodb-semi-consistent.test is owned by Innobase and should not be updated.
File storage/innobase/handler/ha_innodb.cc is owned by Innobase and should not be updated.
File storage/innobase/ibuf/ibuf0ibuf.c is owned by Innobase and should not be updated.
File storage/innobase/include/row0mysql.h is owned by Innobase and should not be updated.
File storage/innobase/include/srv0srv.h is owned by Innobase and should not be updated.
File storage/innobase/include/trx0trx.h is owned by Innobase and should not be updated.
File storage/innobase/include/trx0trx.ic is owned by Innobase and should not be updated.
File storage/innobase/lock/lock0lock.c is owned by Innobase and should not be updated.
File storage/innobase/page/page0cur.c is owned by Innobase and should not be updated.
File storage/innobase/row/row0mysql.c is owned by Innobase and should not be updated.
File storage/innobase/row/row0sel.c is owned by Innobase and should not be updated.
File storage/innobase/srv/srv0srv.c is owned by Innobase and should not be updated.
File storage/innobase/trx/trx0trx.c is owned by Innobase and should not be updated.
(Set env var 'ALLOW_UPDATE_INNOBASE_OWNED' to override.)
key_buffer_size.
The cause of corruption was number overflow when multiplying
two ulong values, number of used keycache blocks with size
of a single block. The result of multiplication exceeded ulong
range (4G) and this lead to incorrectly calculated buffer offset
in the key cache.
The fix is to use size_t for multiplication result.
This patch also fixes pointless cast in safemalloc
(size of allocated block to uint), that creates lot of false
alarm warnings when using big keycache (> 4GB) in debug mode.
In order to define the --slave-load-tmpdir, the init_relay_log_file()
was calling fn_format(MY_PACK_FILENAME) which internally was indirectly
calling strmov_overlapp() (through pack_dirname) and the following
warning message was being printed out while running in Valgrind:
"source and destination overlap in strcpy".
We fixed the issue by removing the flag MY_PACK_FILENAME as it was not
necessary. In a nutshell, with this flag the function fn_format() tried
to replace a directory by either "~", "." or "..". However, we wanted
exactly to remove such strings.
In this patch, we also refactored the functions init_relay_log_file()
and check_temp_dir(). The former was refactored to call the fn_format()
with the flag MY_SAFE_PATH along with the MY_RETURN_REAL_PATH, in order
to avoid issues with long directories and return an absolute path,
respectively. The flag MY_SAFE_UNPACK_FILENAME was removed too as it was
responsible for removing "~", "." or ".." only from the file parameter
and we wanted to remove such strings from the directory parameter in
the fn_format(). This result is stored in an rli variable, which is then
processed by the other function in order to verify if the directory exists
and if we are able to create files in it.
mysql-test/suite/rpl/t/rpl_slave_load_tmpdir_not_exist.test:
Changed the output to make it consistent among different runs.
mysys/mf_format.c:
Replaced a return for DBUG_RETURN.
Bug#44091: libmysqld gets stuck waiting on mutex on initialization
The problem was that libmysqld wasn't enforcing a certain
initialization and deinitialization order for the mysys
library. Another problem was that the global object used
for management of log event handlers (aka LOGGER) wasn't
being prepared for a possible reutilization.
What leads to the hang/crash reported is that a failure
to load the language file triggers a double call of the
cleanup functions, causing an already destroyed mutex to
be used.
The solution is enforce a order on the initialization and
deinitialization of the mysys library within the libmysqld
library and to ensure that the global LOGGER object reset
it's internal state during cleanup.
mysys/my_init.c:
Deinitialize only if initialized already.
sql/log.cc:
Reset state.
Probe definition file is now a separate file that is copied during
build to the correct location, this enforces dependency requirements
correctly and allows builds to work when using the current or separate
build directories
If a sys-var has a base and a block-size>1, and then a
user-supplied value >= minimum ended up below minimum
thanks to block-size alignment, we threw a warning.
This meant for instance that when getting, then setting
the minimum, we'd see a warning. This was needlessly
confusing. (updated patch)
patch enclosed)
One call to my_error_unregister_all() would free pointers, but leave one
pointer to just-freed memory still assigned. That's the bug. Subsequent
calls of this function would try to follow pointers into deallocated,
garbage memory and almost certainly SEGV.
Now, after freeing a linked list, unset the initial pointer.
Took the Xfree implementation (based on the same rewrite as the NDB one)
and added it instead of the current implementation.
Added a macro to make the calls to MD5 more streamlined.
client/mysqlmanager-pwgen.c:
Bug #42434: changed to call the macro
include/my_md5.h:
Bug #42434: use the Xfree implementation
mysys/md5.c:
Bug #42434: use the Xfree implementation
sql/item_strfunc.cc:
Bug #42434: changed to call the macro
sql/table.cc:
Bug #42434: changed to call the macro
tools/mysqlmanager.c:
Bug #42434: changed to call the macro
When disk is full, server may waiting for free space while
writing binlog, relay-log or MyISAM tables. The server will
continue after user have freed some space. But the error
message printed was not quite clear about the how often the
error message is printed, and there will be a delay before
the server continue and user freeing space. And caused users
thinking that the server was hanging forever.
This patch fixed the problem by making the error messages
printed more clear. The error message is split into two part,
the first part will only be printed once, and the second part
will be printed very 10 times.
Message first part:
Disk is full writing '<filename>' (Errcode: <errorno>). Waiting
for someone to free space... (Expect up to 60 secs delay for
server to continue after freeing disk space)
Message second part:
Retry in 60 secs, Message reprinted in 600 secs
in load_defaults()
load_defaults(), my_search_option_files() and
my_print_default_files() utilized a global variable
containing a pointer to thread local memory. This could lead
to race conditions when those functions were called with high
concurrency.
Fixed by changing the interface of the said functions to avoid
the necessity for using a global variable.
Since we cannot change load_defaults() prototype for API
compatibility reasons, it was renamed my_load_defaults().
Now load_defaults() is a thread-unsafe wrapper around
a thread-safe version, my_load_defaults().
mysys/default.c:
1. Added a thread-safe version of load_defaults(), changed
load_defaults() with the old interface to be a thread-unsafe
wrapper around the thread-safe version.
2. Always use a private MEM_ROOT in my_print_default_files,
don't use a global variable.
sql-common/client.c:
Use a thread-safe version of load_defaults().
