When we append data to the binlog file, we use fdatasync() to ensure
the data gets to disk so that crash recovery can work.
Unfortunately there seems to be a bug in ext3/ext4 on linux, so that
fdatasync() does not correctly sync all data when the size of a file
is increased. This causes crash recovery to not work correctly (it
loses transactions from the binlog).
As a work-around, use fsync() for the binlog, not fdatasync(). Since
we are increasing the file size, (correct) fdatasync() will most
likely not be faster than fsync() on any file system, and fsync()
does work correctly on ext3/ext4. This avoids the need to try to
detect if we are running on buggy ext3/ext4.
two tests still fail:
main.innodb_icp and main.range_vs_index_merge_innodb
call records_in_range() with both range ends being open
(which triggers an assert)
DURING SERVER STARTUP
The options parser now correctly checks for ambiguous prefixes in
enumerated variables and emits an error when the value supplied is
ambiguous.
No test added since mysql-test-run.pl can't handle server startup
failures as an expected state.
'MAX_BINLOG_CACHE_SIZE' ERROR
Problem:
=======
MySQL returns following error in win64.
"ERROR 1197 (HY000): Multi-statement transaction required more than
'max_binlog_cache_size' bytes of storage; increase this mysqld variable
and try again" when user tries to load >4G file even if
max_binlog_cache_size set to maximum value. On Linux everything
works fine.
Analysis:
========
The `max_binlog_cache_size' variable is of type `ulonglong'. This
value is set to `ULONGLONG_MAX' at the time of server start up. The
above value is stored in an intermediate variable named
`saved_max_binlog_cache_size' which is of type `ulong'. In visual
c++ complier the `ulong' type is of 4bytes in size and hence the value
is getting truncated to '4GB' and the cache is not able to grow beyond
4GB size. The same limitation is observed with
"max_binlog_stmt_cache_size" as well. Similar fix has been applied.
Fix:
===
As part of fix the type "ulong" is replaced with "my_off_t" which is of
type "ulonglong".
mysys/mf_iocache.c:
Added debug statement to simulate a scenario where the cache
file's current position is set to >4GB
sql/log.cc:
Replaced the type of `saved_max_binlog_cache_size' from "ulong" to
"my_off_t", which is a type def for "ulonglong".
Problem:
=======
The return value from my_b_write is ignored by: `my_b_write_quoted',
`my_b_write_bit',`Query_log_event::print_query_header'
Most callers of `my_b_printf' ignore the return value. `log_event.cc'
has many calls to it.
Analysis:
========
`my_b_write' is used to write data into a file. If the write fails it
sets appropriate error number and error message through my_error()
function call and sets the IO_CACHE::error == -1.
`my_b_printf' function is also used to write data into a file, it
internally invokes my_b_write to do the write operation. Upon
success it returns number of characters written to file and on error
it returns -1 and sets the error through my_error() and also sets
IO_CACHE::error == -1. Most of the event specific print functions
for example `Create_file_log_event::print', `Execute_load_log_event::print'
etc are the ones which make several calls to the above two functions and
they do not check for the return value after the 'print' call. All the above
mentioned abuse cases deal with the client side.
Fix:
===
As part of bug fix a check for IO_CACHE::error == -1 has been added at
a very high level after the call to the 'print' function. There are
few more places where the return value of "my_b_write" is ignored
those are mentioned below.
+++ mysys/mf_iocache2.c 2012-06-04 07:03:15 +0000
@@ -430,7 +430,8 @@
memset(buffz, '0', minimum_width - length2);
else
memset(buffz, ' ', minimum_width - length2);
- my_b_write(info, buffz, minimum_width - length2);
+++ sql/log.cc 2012-06-08 09:04:46 +0000
@@ -2388,7 +2388,12 @@
{
end= strxmov(buff, "# administrator command: ", NullS);
buff_len= (ulong) (end - buff);
- my_b_write(&log_file, (uchar*) buff, buff_len);
At these places appropriate return value handlers have been added.
client/mysqlbinlog.cc:
check for IO_CACHE::error == -1 has been added after the call to
the event specific print functions
mysys/mf_iocache2.c:
Added handler to check the written value of `my_b_write'
sql/log.cc:
Added handler to check the written value of `my_b_write'
sql/log_event.cc:
Added error simulation statements in `Create_file_log_event::print`
and `Execute_load_query_log_event::print'
sql/rpl_utility.h:
Removed the extra ';'
HANDLE_FATAL_SIGNAL IN STRNLEN
Fixed the following bounds checking problems :
1. in check_if_legal_filename() make sure the null terminated
string is long enough before accessing the bytes in it.
Prevents pottential read-past-buffer-end
2. in my_wc_mb_filename() of the filename charset check
for the end of the destination buffer before sending single
byte characters into it.
Prevents write-past-end-of-buffer (and garbaling stack in
the cases reported here) errors.
Added test cases.
- Changed output to be error "error-text" instead of error - error-text
extra/perror.c:
Move my_handler_errors.h into include
include/my_handler_errors.h:
Move my_handler_errors.h into include
mysql-test/r/errors.result:
Updated result
mysql-test/r/innodb_mysql_sync.result:
Updated result
mysql-test/r/myisam-system.result:
Updated result
mysql-test/r/myisampack.result:
Updated result
mysql-test/r/partition_innodb_plugin.result:
Updated result
mysql-test/r/ps_1general.result:
Updated result
mysql-test/r/trigger.result:
Updated result
mysql-test/r/type_bit.result:
Updated result
mysql-test/r/type_bit_innodb.result:
Updated result
mysql-test/r/type_blob.result:
Updated result
mysql-test/suite/archive/archive.result:
Updated result
mysql-test/suite/binlog/r/binlog_index.result:
Updated result
mysql-test/suite/binlog/r/binlog_ioerr.result:
Updated result
mysql-test/suite/csv/csv.result:
Updated result
mysql-test/suite/engines/iuds/r/type_bit_iuds.result:
Updated result
mysql-test/suite/federated/federated_bug_35333.result:
Updated result
mysql-test/suite/innodb/r/innodb-create-options.result:
Updated result
mysql-test/suite/innodb/r/innodb-index.result:
Updated result
mysql-test/suite/innodb/r/innodb-zip.result:
Updated result
mysql-test/suite/innodb/r/innodb.result:
Updated result
mysql-test/suite/innodb/r/innodb_bug13635833.result:
Updated result
mysql-test/suite/innodb/r/innodb_bug21704.result:
Updated result
mysql-test/suite/innodb/r/innodb_bug46000.result:
Updated result
mysql-test/suite/parts/r/partition_bit_innodb.result:
Updated result
mysql-test/suite/parts/r/partition_bit_myisam.result:
Updated result
mysql-test/suite/percona/percona_innodb_fake_changes.result:
Updated result
mysql-test/suite/perfschema/r/misc.result:
Updated result
mysql-test/suite/perfschema/r/privilege.result:
Updated result
mysql-test/suite/rpl/r/rpl_EE_err.result:
Updated result
mysql-test/suite/rpl/r/rpl_binlog_errors.result:
Updated result
mysql-test/suite/rpl/r/rpl_drop_db.result:
Updated result
sql/share/errmsg-utf8.txt:
Removed 'column' from error text that was used in different context
strings/my_vsnprintf.c:
Move my_handler_errors.h into include
Minor cleanups
Changed output of %M to be error "error-text" instead of error - error-text
unittest/mysys/my_vsnprintf-t.c:
Updated error text
One of the reported problems manifested itself in the scenario when one
thread tried to to get statistics on a key cache while the second thread
had not finished initialization of the key cache structure yet.
The problem was resolved by forcing serialization of such operations
on key caches.
To serialize function calls to perform certain operations over a key cache
a new mutex associated with the key cache now is used. It is stored in the
field op_lock of the KEY_CACHE structure. It is locked when the operation
is performed. Some of the serialized key cache operations utilize calls
for other key cache operations. To avoid recursive locking of op_lock
the new functions that perform the operations of key cache initialization,
destruction and re-partitioning with an additional parameter were introduced.
The parameter says whether the operation over op_lock are to be performed or
are to be omitted. The old functions for the operations of key cache
initialization, destruction,and re-partitioning now just call the
corresponding new functions with the additional parameter set to true
requesting to use op_lock while all other calls of these new function
have this parameter set to false.
Another problem reported in the bug entry concerned the operation of
assigning an index to a key cache. This operation can be called
while the key cache structures are not initialized yet. In this
case any call of flush_key_blocks() should return without any actions.
No test case is provided with this patch.
- Adding %M my_sprintf() modifier that prints error number - system-error-text
- Modified mysys, mysql_client and SQL error messages to use %M instead of %d
- Added my_strerror()
Updated handler errors to 5.6 error numbers
Updated text for a few error messages (to match 5.6)
Increased length of command name in error output
extra/comp_err.c:
Added support for %M
include/my_base.h:
Updated handler errors to 5.6 error numbers
include/my_sys.h:
Added my_strerror()
libmysql/errmsg.c:
Updated error messages to use %M
mysql-test/r/errors.result:
Updated result as error message have changed
mysql-test/r/innodb_mysql_sync.result:
Updated result with text for errno
mysql-test/r/myisam-system.result:
Updated result with text for errno
mysql-test/r/myisam.result:
Updated result as error message have changed
mysql-test/r/myisampack.result:
Updated result with text for errno
mysql-test/r/mysql.result:
Updated result with text for errno
mysql-test/r/mysql_upgrade.result:
Updated result with text for errno
mysql-test/r/partition_datatype.result:
Updated result as error message have changed
mysql-test/r/partition_innodb_plugin.result:
Updated result with text for errno
mysql-test/r/ps_1general.result:
Updated result with text for errno
mysql-test/r/trigger.result:
Updated result with text for errno
mysql-test/r/type_bit.result:
Updated result as error message have changed
mysql-test/r/type_bit_innodb.result:
Updated result as error message have changed
mysql-test/r/type_blob.result:
Updated result as error message have changed
mysql-test/suite/archive/archive.result:
Updated result with text for errno
mysql-test/suite/binlog/r/binlog_index.result:
Updated result with text for errno
mysql-test/suite/binlog/r/binlog_ioerr.result:
Updated result with text for errno
mysql-test/suite/csv/csv.result:
Updated result with text for errno
mysql-test/suite/federated/federated_bug_35333.result:
Updated result with text for errno
mysql-test/suite/innodb/r/innodb-create-options.result:
Updated result with text for errno
mysql-test/suite/innodb/r/innodb-index.result:
Updated result with text for errno
mysql-test/suite/innodb/r/innodb-zip.result:
Updated result as error message have changed
mysql-test/suite/innodb/r/innodb.result:
Updated result with text for errno
mysql-test/suite/innodb/r/innodb_bug21704.result:
Updated result with text for errno
mysql-test/suite/innodb/r/innodb_bug46000.result:
Updated result with text for errno
mysql-test/suite/innodb/r/innodb_bug53591.result:
Updated result as error message have changed
mysql-test/suite/innodb/r/innodb_corrupt_bit.result:
New error numbers
mysql-test/suite/innodb/r/innodb_prefix_index_liftedlimit.result:
Updated result as error message have changed
mysql-test/suite/innodb/t/innodb-create-options.test:
Added regexp to avoid system error text
mysql-test/suite/innodb/t/innodb-zip.test:
Added regexp to avoid system error text
mysql-test/suite/maria/maria-recovery2.result:
Updated supression rule
mysql-test/suite/maria/maria-recovery2.test:
Updated supression rule
mysql-test/suite/maria/maria.result:
Updated result as error message have changed
mysql-test/suite/parts/r/partition_bit_innodb.result:
Updated result as error message have changed
mysql-test/suite/parts/r/partition_bit_myisam.result:
Updated result as error message have changed
mysql-test/suite/percona/percona_innodb_fake_changes.result:
Updated result with text for errno
mysql-test/suite/perfschema/r/dml_cond_instances.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_events_waits_current.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_events_waits_history.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_events_waits_history_long.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_ews_by_instance.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_ews_by_thread_by_event_name.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_ews_global_by_event_name.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_file_instances.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_file_summary_by_event_name.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_file_summary_by_instance.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_mutex_instances.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_performance_timers.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_rwlock_instances.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_threads.result:
Updated result as error message have changed
mysql-test/suite/perfschema/r/misc.result:
Updated result with text for errno
mysql-test/suite/perfschema/r/privilege.result:
Updated result with text for errno
mysql-test/suite/rpl/r/rpl_EE_err.result:
Updated result with text for errno
mysql-test/suite/rpl/r/rpl_binlog_errors.result:
Updated result with text for errno
mysql-test/suite/rpl/r/rpl_drop_db.result:
Updated result with text for errno
mysys/errors.c:
Updated error messages to use %M
Changed all errors to use Errcode: consistenly
mysys/my_handler_errors.h:
Updated handler errors to 5.6 error numbers
sql/share/errmsg-utf8.txt:
Updated error messages to use %M
sql/sys_vars.cc:
Added error number to ER_EVENT_SET_VAR_ERROR
strings/my_vsnprintf.c:
Added %M my_sprintf() modifier that prints error number - system-error-text
Simplify code
Movied common code to function
Removed some casts that was not necessary when reading integer/unsigned int stored in longlong
Added my_strerror()
unittest/mysys/my_vsnprintf-t.c:
Added testing of %M
Problem: mysqlbinlog exits without any error code in case of
file write error. It is because of the fact that the calls
to Log_event::print() method does not return a value and the
thus any error were being ignored.
Resolution: We resolve this problem by checking for the
IO_CACHE::error == -1 after every call to Log_event:: print()
and terminating the further execution.
client/mysqlbinlog.cc:
- handled error conditions during event->print() calls
- added check for error in end_io_cache()
mysys/my_write.c:
Added debug code to simulate file write error.
error returned will be ENOSPC=> error no space on the disk
sql/log_event.cc:
Added debug code to simulate file write error, by reducing the size of io cache.
Fixed failing tests in sys_vars as we have now stricter checking of setting of variables.
mysql-test/suite/sys_vars/r/innodb_adaptive_flushing_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_adaptive_hash_index_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_large_prefix_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_random_read_ahead_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_stats_on_metadata_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_strict_mode_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_support_xa_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_table_locks_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/rpl_semi_sync_master_enabled_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/rpl_semi_sync_slave_enabled_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_adaptive_flushing_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_adaptive_hash_index_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_large_prefix_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_random_read_ahead_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_stats_on_metadata_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_strict_mode_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_support_xa_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_table_locks_basic.test:
One can now only assign 0 or 1 to boolean variables
mysys/my_getsystime.c:
Merge + fixed bug that __NR_clock_gettime didn't work in 5.5
