Write an additional warning message to the server log,
explaining why a sort operation is aborted.
The output in mysqld.err will look something like:
110127 15:07:54 [ERROR] mysqld: Sort aborted: Out of memory (Needed 24 bytes)
110127 15:07:54 [ERROR] mysqld: Out of sort memory, consider increasing server sort buffer size
110127 15:07:54 [ERROR] mysqld: Sort aborted: Out of sort memory, consider increasing server sort buffer size
110127 15:07:54 [ERROR] mysqld: Sort aborted: Incorrect number of arguments for FUNCTION test.f1; expected 0, got 1
If --log-warn=2 is enabled, we output information about host/user/query as well.
include/my_sys.h:
Update comment for ME_NOREFRESH
mysql-test/include/mtr_warnings.sql:
Remove global filtering of "Out of sort memory", let each individual test set it instead.
mysql-test/r/filesort_debug.result:
New test case.
mysql-test/r/order_by.result:
Ignore "Out of memory" for this test.
mysql-test/t/filesort_debug.test:
New test case.
mysql-test/t/order_by.test:
Ignore "Out of memory" for this test.
sql/filesort.cc:
Output an explanation using the error message from the THD Diagnostics_area.
sql/protocol.cc:
Do not DBUG_RETURN(function_call_with DBUG_RETURN)
as it messes up the call stack in the debug output.
sql/share/errmsg-utf8.txt:
Change error message for "Out of sort memory"
sql/unireg.h:
Remove unused/confusing ERRMAPP macro.
Write an additional warning message to the server log,
explaining why a sort operation is aborted.
The output in mysqld.err will look something like:
110127 15:07:54 [ERROR] mysqld: Sort aborted: Out of memory (Needed 24 bytes)
110127 15:07:54 [ERROR] mysqld: Out of sort memory, consider increasing server sort buffer size
110127 15:07:54 [ERROR] mysqld: Sort aborted: Out of sort memory, consider increasing server sort buffer size
110127 15:07:54 [ERROR] mysqld: Sort aborted: Incorrect number of arguments for FUNCTION test.f1; expected 0, got 1
If --log-warn=2 is enabled, we output information about host/user/query as well.
and 'THREAD_SAFE_CLIENT'.
As of MySQL 5.5, we no longer support non-threaded
builds. This patch removes all references to the
obsolete THREAD and THREAD_SAFE_CLIENT preprocessor
symbols. These were used to distinguish between
threaded and non-threaded builds.
and 'THREAD_SAFE_CLIENT'.
As of MySQL 5.5, we no longer support non-threaded
builds. This patch removes all references to the
obsolete THREAD and THREAD_SAFE_CLIENT preprocessor
symbols. These were used to distinguish between
threaded and non-threaded builds.
From a user perspective, the problem is that a FLUSH LOGS or SIGHUP
signal could end up associating the stdout and stderr to random
files. In the case of this bug report, the streams would end up
associated to InnoDB ibd files.
The freopen(3) function is not thread-safe on FreeBSD. What this
means is that if another thread calls open(2) during freopen()
is executing that another thread's fd returned by open(2) may get
re-associated with the file being passed to freopen(3). See FreeBSD
PR number 79887 for reference:
http://www.freebsd.org/cgi/query-pr.cgi?pr=79887
This problem is worked around by substituting a internal hook within
the FILE structure. This avoids the loss of atomicity by not having
the original fd closed before its duplicated.
Patch based on the original work by Vasil Dimov.
include/my_sys.h:
Export my_freopen.
mysys/my_fopen.c:
Add a my_freopen abstraction to workaround bugs in specific OSes.
Add a prototype for getosreldate() as older FreeBSD versions did
not define one.
sql/log.cc:
Move freopen abstraction code over to mysys.
The streams are now only reopened for writing.
From a user perspective, the problem is that a FLUSH LOGS or SIGHUP
signal could end up associating the stdout and stderr to random
files. In the case of this bug report, the streams would end up
associated to InnoDB ibd files.
The freopen(3) function is not thread-safe on FreeBSD. What this
means is that if another thread calls open(2) during freopen()
is executing that another thread's fd returned by open(2) may get
re-associated with the file being passed to freopen(3). See FreeBSD
PR number 79887 for reference:
http://www.freebsd.org/cgi/query-pr.cgi?pr=79887
This problem is worked around by substituting a internal hook within
the FILE structure. This avoids the loss of atomicity by not having
the original fd closed before its duplicated.
Patch based on the original work by Vasil Dimov.
After fix of bug#25192, load_defaults() will add an args separator
to distinguish options loaded from configure files from that provided
in the command line. One problem of this is that the args separator
would be added no matter the application need it or not.
Fixed the problem by adding an option:
bool my_getopt_use_args_separator;
to control whether the separator will be added or not. And also
added functions:
bool my_getopt_is_args_separator(const char* arg);
to check if the argument is the separator or not.
After fix of bug#25192, load_defaults() will add an args separator
to distinguish options loaded from configure files from that provided
in the command line. One problem of this is that the args separator
would be added no matter the application need it or not.
Fixed the problem by adding an option:
bool my_getopt_use_args_separator;
to control whether the separator will be added or not. And also
added functions:
bool my_getopt_is_args_separator(const char* arg);
to check if the argument is the separator or not.
- Removed files specific to compiling on OS/2
- Removed files specific to SCO Unix packaging
- Removed "libmysqld/copyright", text is included in documentation
- Removed LaTeX headers for NDB Doxygen documentation
- Removed obsolete NDB files
- Removed "mkisofs" binaries
- Removed the "cvs2cl.pl" script
- Changed a few GPL texts to use "program" instead of "library"
- Removed files specific to compiling on OS/2
- Removed files specific to SCO Unix packaging
- Removed "libmysqld/copyright", text is included in documentation
- Removed LaTeX headers for NDB Doxygen documentation
- Removed obsolete NDB files
- Removed "mkisofs" binaries
- Removed the "cvs2cl.pl" script
- Changed a few GPL texts to use "program" instead of "library"
- This allows us to detect missing my_afree() calls and also find overruns (when running with valgrind) to alloca() areas.
- Added missing my_afree() calls
- Fixed wrong call to my_afree()
include/my_sys.h:
When compiling with valgrind, change my_alloca() to use my_malloc()
mysql-test/suite/innodb/t/innodb_bug57255.test:
Speed up taste case (patch from Stewart Smith)
mysql-test/suite/innodb_plugin/t/innodb_bug57255.test:
Speed up taste case (patch from Stewart Smith)
sql/ha_partition.cc:
Removed casts from my_afree()
sql/opt_range.cc:
Add missing my_afree() calls.
storage/maria/ma_rt_split.c:
Fixed wrong parameter to my_afree()
Removed wrong call to translog_buffer_unlock() that caused 'unlocking not locked mutex' failure in Aria log handler.
extra/replace.c:
Updated call to my_redel()
include/maria.h:
Updated prototype for maria_change_to_newfile
include/my_sys.h:
Updated prototypes
Added my_create_backup_name
include/myisam.h:
Updated prototypes
include/myisamchk.h:
Added 'backup_time' to st_handler_check_param to be able to generate same name for backuped data and index file
mysys/my_redel.c:
Added time_t option to my_redel() to be able to generate same backup extensions for many files
sql/handler.cc:
Added start_time to st_ha_check_opt
sql/handler.h:
Added start_time to HA_CHECK_OPT
sql/mysqld.cc:
Added option BACKUP_ALL to --myisam-recover
storage/maria/ha_maria.cc:
Remember start time for repair
storage/maria/ma_check.c:
Use remembered start time for backup file names
Removed some dead code
storage/maria/ma_loghandler.c:
Removed wrong call to translog_buffer_unlock() that caused 'unlocking not locked mutex' failure in log handler.
storage/maria/maria_chk.c:
Removed dead code (O_NEW_INDEX was never set)
Report if table was 'crashed on repair'
storage/maria/maria_pack.c:
Updated parameters to my_redel()
storage/myisam/ha_myisam.cc:
Added recover option BACKUP_ALL to get a backup of the index file before it's repaired.
Print information to log if we make a backup of data as part of repair.
storage/myisam/ha_myisam.h:
Added HA_RECOVER_FULL_BACKUP
storage/myisam/mi_check.c:
Use remembered start time for backup file names
Added mi_make_backup_of_index()
storage/myisam/myisamchk.c:
Removed dead code (O_NEW_INDEX was never set)
Report if table was 'crashed on repair'
storage/myisam/myisampack.c:
Updated call to my_redel()
After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the
stmt-cache and the trx-cache. This means that the resource used is twice the amount
expected/defined by the user.
The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used
and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the
trx-cache is used. This behavior does not allow to distinguish which cache may be harming
performance due to the extra disk accesses and needs to have its in-memory cache
increased.
To fix the problem, we introduced two new options and status variables related to the
stmt-cache:
Options:
. binlog_stmt_cache_size
. max_binlog_stmt_cache_size
Status Variables:
. binlog_stmt_cache_use
. binlog_stmt_cache_disk_use
So there are
. binlog_cache_size that defines the size of the transactional cache for
updates to transactional engines for the binary log.
. binlog_stmt_cache_size that defines the size of the statement cache for
updates to non-transactional engines for the binary log.
. max_binlog_cache_size that sets the total size of the transactional
cache.
. max_binlog_stmt_cache_size that sets the total size of the statement
cache.
. binlog_cache_use that identifies the number of transactions that used the
temporary transactional binary log cache.
. binlog_cache_disk_use that identifies the number of transactions that used
the temporary transactional binary log cache but that exceeded the value of
binlog_cache_size.
. binlog_stmt_cache_use that identifies the number of statements that used the
temporary non-transactional binary log cache.
. binlog_stmt_cache_disk_use that identifies the number of statements that used
the temporary non-transactional binary log cache but that exceeded the value of
binlog_stmt_cache_size.
include/my_sys.h:
Updated message on disk_writes' usage.
mysql-test/extra/binlog_tests/binlog_cache_stat.test:
Updated the test case and added code to check the new status variables
binlog_stmt_cache_use and binlog_stmt_cache_disk_use.
mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
mysql-test/r/mysqld--help-notwin.result:
Updated the result file.
mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result:
Updated the result file.
mysql-test/suite/binlog/r/binlog_row_cache_stat.result:
Updated the result file.
mysql-test/suite/binlog/r/binlog_stm_cache_stat.result:
Updated the result file.
mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result:
Update the result file.
mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result:
Update the result file.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
Updated the result file.
mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc:
Added a test case to check the binlog_stmt_cache_size.
mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result:
Updated the result file.
mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result:
Updated the result file.
mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result:
Updated the result file.
mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test:
Added a test case to check the binlog_stmt_cache_size.
mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test:
Added a test case to check the binlog_stmt_cache_size.
mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt:
Removed because there is no test case max_binlog_cache_size_func.
mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test:
Added a test case to check the system variable max_binlog_stmt_cache_size.
sql/log.cc:
There two main changes in here:
. Changed the set_write_error() as an error message is set according
to the type of the cache.
. Created the function set_binlog_cache_info where references to the
appropriate status and system variables are set and the server can
smoothly compute statistics and set the maximum size for each cache.
sql/log.h:
Changed the signature of the function in order to identify the error message
to be printed out as there is a different error code for each type of cache.
sql/mysqld.cc:
Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use.
sql/mysqld.h:
Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
sql/share/errmsg-utf8.txt:
Added new error message related to the statement cache.
sql/sys_vars.cc:
Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the
stmt-cache and the trx-cache. This means that the resource used is twice the amount
expected/defined by the user.
The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used
and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the
trx-cache is used. This behavior does not allow to distinguish which cache may be harming
performance due to the extra disk accesses and needs to have its in-memory cache
increased.
To fix the problem, we introduced two new options and status variables related to the
stmt-cache:
Options:
. binlog_stmt_cache_size
. max_binlog_stmt_cache_size
Status Variables:
. binlog_stmt_cache_use
. binlog_stmt_cache_disk_use
So there are
. binlog_cache_size that defines the size of the transactional cache for
updates to transactional engines for the binary log.
. binlog_stmt_cache_size that defines the size of the statement cache for
updates to non-transactional engines for the binary log.
. max_binlog_cache_size that sets the total size of the transactional
cache.
. max_binlog_stmt_cache_size that sets the total size of the statement
cache.
. binlog_cache_use that identifies the number of transactions that used the
temporary transactional binary log cache.
. binlog_cache_disk_use that identifies the number of transactions that used
the temporary transactional binary log cache but that exceeded the value of
binlog_cache_size.
. binlog_stmt_cache_use that identifies the number of statements that used the
temporary non-transactional binary log cache.
. binlog_stmt_cache_disk_use that identifies the number of statements that used
the temporary non-transactional binary log cache but that exceeded the value of
binlog_stmt_cache_size.
If a relative path is supplied to option --defaults-file or
--defaults-extra-file, the server will crash when executing
an INSTALL PLUGIN command. The reason is that the defaults
file is initially read relative the current working directory
when the server is started, but when INSTALL PLUGIN is executed,
the server has changed working directory to the data directory.
Since there is no check that the call to my_load_defaults()
inside mysql_install_plugin(), the subsequence call to
free_defaults() will crash the server.
This patch fixes the problem by:
- Prepending the current working directory to the file name when
a relative path is given to the --defaults-file or --defaults-
extra-file option the first time my_load_defaults() is called,
which is just after the server has started in main().
- Adding a check of the return value of my_load_defaults() inside
mysql_install_plugin() and aborting command (with an error) if
an error is returned.
- It also adds a check of the return value for load_defaults in
lib_sql.cc for the embedded server since that was missing.
To test that the relative files for the options --defaults-file and
--defaults-extra-file is handled properly, mysql-test-run.pl is also
changed to not add a --defaults-file option if one is provided in the
tests *.opt file.
If a relative path is supplied to option --defaults-file or
--defaults-extra-file, the server will crash when executing
an INSTALL PLUGIN command. The reason is that the defaults
file is initially read relative the current working directory
when the server is started, but when INSTALL PLUGIN is executed,
the server has changed working directory to the data directory.
Since there is no check that the call to my_load_defaults()
inside mysql_install_plugin(), the subsequence call to
free_defaults() will crash the server.
This patch fixes the problem by:
- Prepending the current working directory to the file name when
a relative path is given to the --defaults-file or --defaults-
extra-file option the first time my_load_defaults() is called,
which is just after the server has started in main().
- Adding a check of the return value of my_load_defaults() inside
mysql_install_plugin() and aborting command (with an error) if
an error is returned.
- It also adds a check of the return value for load_defaults in
lib_sql.cc for the embedded server since that was missing.
To test that the relative files for the options --defaults-file and
--defaults-extra-file is handled properly, mysql-test-run.pl is also
changed to not add a --defaults-file option if one is provided in the
tests *.opt file.
Fixed LP#613418 (M)aria recovery failure: ma_key_recover.c:981: _ma_apply_redo_index: Assertion `check_page_length == page_length' failed
include/my_sys.h:
Added my_disable_sync
mysql-test/mysql-test-run.pl:
Added --sync-sys=0 to run test suite faster
mysys/my_static.c:
Added my_disable_sync
mysys/my_sync.c:
Added my_disable_sync
sql/mysqld.cc:
Added -sync-sys=0 option for mysqld to skip sync() calls for faster testing
storage/maria/ma_key_recover.c:
More DBUG_ASSERT()
Added logging of KEY_OP_DEBUG to make examening of logs easier
Fixed testing of page length in recovery to ensure we don't overwrite checksum (previous tests was too relaxed)
Fixed bug in recovery logging of split pages which caused failure during recovery:
- Length was not adjusted properly for pages to be split
- Added KEY_OP_MAX_PAGELENGTH to tell recovery that page is now full length
- This fixed LP#613418
storage/maria/ma_key_recover.h:
Changed prototype for ma_log_change() for KEY_OP_DEBUG
storage/maria/ma_loghandler.h:
Added new enums for better debugging of recovery logs
storage/maria/ma_rt_index.c:
Added debugging information to calls to ma_log_change()
storage/maria/ma_write.c:
Added debugging information to calls to ma_log_change() and ma_log_split()
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
