mariadb/mysql-test/extra/binlog_tests/binlog_cache_stat.test

252 lines
13 KiB
Text
Raw Normal View History

BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
# Embedded server doesn't support binlog
-- source include/not_embedded.inc
-- source include/have_innodb.inc
# Creating tables
--disable_warnings
drop table if exists t1, t2;
--enable_warnings
create table t1 (a int) engine=innodb;
create table t2 (a int) engine=myisam;
#
# This test checks binlog_cache_use and binlog_cache_disk_use when
# transactions are committed and after when they are aborted.
#
#
# Checking commit.
#
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Preparing the enviroment to check commit and its effect on status variables.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Expected: binlog_cache_use = 0, binlog_cache_disk_use = 0.
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Expected: binlog_stmt_cache_use = 0, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
flush status;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $exp_cache= 0;
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 0;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 0;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Transactional changes which are long enough so they will be flushed to disk...
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Expected: binlog_cache_use = 1, binlog_cache_disk_use = 1.
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Expected: binlog_stmt_cache_use = 0, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
let $1=2000;
disable_query_log;
begin;
while ($1)
{
eval insert into t1 values( $1 );
dec $1;
}
commit;
enable_query_log;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $exp_cache= 1;
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 0;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Transactional changes which should not be flushed to disk and so should not
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** increase either binlog_cache_disk_use or binlog_stmt_cache_disk_use.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Expected: binlog_cache_use = 2, binlog_cache_disk_use = 1.
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Expected: binlog_stmt_cache_use = 0, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
begin;
insert into t1 values( 1 );
commit;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $exp_cache= 2;
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 0;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Non-Transactional changes which should not be flushed to disk and so should not
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** increase either binlog_cache_disk_use or binlog_stmt_cache_disk_use.
--echo **** Expected: binlog_cache_use = 2, binlog_cache_disk_use = 1.
--echo **** Expected: binlog_stmt_cache_use = 1, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
begin;
insert into t2 values( 1 );
commit;
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_cache= 2;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 1;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Mixed changes which should not be flushed to disk and so should not
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** increase either binlog_cache_disk_use or binlog_stmt_cache_disk_use.
--echo **** Expected: binlog_cache_use = 3, binlog_cache_disk_use = 1.
--echo **** Expected: binlog_stmt_cache_use = 2, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
begin;
insert into t1 values( 1 );
insert into t2 values( 1 );
commit;
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_cache= 3;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 2;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
#
# Checking abort.
#
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Preparing the enviroment to check abort and its effect on the status variables.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Expected: binlog_cache_use = 0, binlog_cache_disk_use = 0.
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Expected: binlog_stmt_cache_use = 0, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
flush status;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $exp_cache= 0;
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 0;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 0;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Transactional changes which are long enough so they will be flushed to disk...
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Expected: binlog_cache_use = 1, binlog_cache_disk_use = 1.
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Expected: binlog_stmt_cache_use = 0, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
let $1=2000;
disable_query_log;
begin;
while ($1)
{
eval insert into t1 values( $1 );
dec $1;
}
rollback;
enable_query_log;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $exp_cache= 1;
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 0;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Transactional changes which should not be flushed to disk and so should not
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** increase either binlog_cache_disk_use or binlog_stmt_cache_disk_use.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Expected: binlog_cache_use = 2, binlog_cache_disk_use = 1.
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** Expected: binlog_stmt_cache_use = 0, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
begin;
insert into t1 values( 1 );
rollback;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $exp_cache= 2;
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 0;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Non-Transactional changes which should not be flushed to disk and so should not
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** increase either binlog_cache_disk_use or binlog_stmt_cache_disk_use.
--echo **** Expected: binlog_cache_use = 2, binlog_cache_disk_use = 1.
--echo **** Expected: binlog_stmt_cache_use = 1, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
begin;
insert into t2 values( 1 );
rollback;
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_cache= 2;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 1;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
--echo **** Mixed changes which should not be flushed to disk and so should not
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
--echo **** increase either binlog_cache_disk_use or binlog_stmt_cache_disk_use.
--echo **** Expected: binlog_cache_use = 3, binlog_cache_disk_use = 1.
--echo **** Expected: binlog_stmt_cache_use = 2, binlog_stmt_cache_disk_use = 0.
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
begin;
insert into t1 values( 1 );
insert into t2 values( 1 );
rollback;
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_cache= 3;
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
let $got_cache= query_get_value(show status like "binlog_cache_use", Value, 1);
let $exp_disk= 1;
let $got_disk= query_get_value(show status like "binlog_cache_disk_use", Value, 1);
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
let $exp_stmt_cache= 2;
let $got_stmt_cache= query_get_value(show status like "binlog_stmt_cache_use", Value, 1);
let $exp_stmt_disk= 0;
let $got_stmt_disk= query_get_value(show status like "binlog_stmt_cache_disk_use", Value, 1);
if (`SELECT $got_cache <> $exp_cache || $got_disk <> $exp_disk || $got_stmt_cache <> $exp_stmt_cache || $got_stmt_disk <> $exp_stmt_disk`)
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
{
BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the stmt-cache and the trx-cache. This means that the resource used is twice the amount expected/defined by the user. The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the trx-cache is used. This behavior does not allow to distinguish which cache may be harming performance due to the extra disk accesses and needs to have its in-memory cache increased. To fix the problem, we introduced two new options and status variables related to the stmt-cache: Options: . binlog_stmt_cache_size . max_binlog_stmt_cache_size Status Variables: . binlog_stmt_cache_use . binlog_stmt_cache_disk_use So there are . binlog_cache_size that defines the size of the transactional cache for updates to transactional engines for the binary log. . binlog_stmt_cache_size that defines the size of the statement cache for updates to non-transactional engines for the binary log. . max_binlog_cache_size that sets the total size of the transactional cache. . max_binlog_stmt_cache_size that sets the total size of the statement cache. . binlog_cache_use that identifies the number of transactions that used the temporary transactional binary log cache. . binlog_cache_disk_use that identifies the number of transactions that used the temporary transactional binary log cache but that exceeded the value of binlog_cache_size. . binlog_stmt_cache_use that identifies the number of statements that used the temporary non-transactional binary log cache. . binlog_stmt_cache_disk_use that identifies the number of statements that used the temporary non-transactional binary log cache but that exceeded the value of binlog_stmt_cache_size. include/my_sys.h: Updated message on disk_writes' usage. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case and added code to check the new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test: Updated the test case to use the new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. mysql-test/r/mysqld--help-notwin.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/rpl/r/rpl_mixed_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_row_binlog_max_cache_size.result: Update the result file. mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result: Updated the result file. mysql-test/suite/sys_vars/inc/binlog_stmt_cache_size_basic.inc: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_32.result: Updated the result file. mysql-test/suite/sys_vars/r/binlog_stmt_cache_size_basic_64.result: Updated the result file. mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result: Updated the result file. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_32.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/binlog_stmt_cache_size_basic_64.test: Added a test case to check the binlog_stmt_cache_size. mysql-test/suite/sys_vars/t/max_binlog_cache_size_func-master.opt: Removed because there is no test case max_binlog_cache_size_func. mysql-test/suite/sys_vars/t/max_binlog_stmt_cache_size_basic.test: Added a test case to check the system variable max_binlog_stmt_cache_size. sql/log.cc: There two main changes in here: . Changed the set_write_error() as an error message is set according to the type of the cache. . Created the function set_binlog_cache_info where references to the appropriate status and system variables are set and the server can smoothly compute statistics and set the maximum size for each cache. sql/log.h: Changed the signature of the function in order to identify the error message to be printed out as there is a different error code for each type of cache. sql/mysqld.cc: Added new status variables binlog_stmt_cache_use and binlog_stmt_cache_disk_use. sql/mysqld.h: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size. sql/share/errmsg-utf8.txt: Added new error message related to the statement cache. sql/sys_vars.cc: Added new system variables max_binlog_stmt_cache_size and binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
-- echo "Expected: binlog_cache_use = $exp_cache, binlog_cache_disk_use = $exp_disk but got binlog_cache_use = $got_cache, binlog_cache_disk_use = $got_disk"
-- echo "Expected: binlog_stmt_cache_use = $exp_stmt_cache, binlog_stmt_cache_disk_use = $exp_stmt_disk but got binlog_stmt_cache_use = $got_stmt_cache, binlog_stmt_cache_disk_use = $got_stmt_disk"
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. Although not explicitly mentioned in the bug, the binlog_cache_disk_use presents the same problem. The binlog_cache_use and binlog_cache_disk_use are status variables that are incremented when transactional (i.e. trx-cache) or non-transactional (i.e. stmt-cache) changes are committed. They are used to compute the ratio between the use of disk and memory while gathering updates for a transaction. The problem reported is fixed by avoiding incrementing the binlog_cache_use and binlog_cache_disk_use if a cache is empty. We also have decided to increment both variables when a cache is truncated as the cache is used although its content is discarded and is not written to the binary log. In this patch, we take the opportunity to re-organize the code around the function binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log.cc: There are four changes in here: . Computed statistics on binlog_cache_use and binlog_cache_disk_use while resting the cache. . Refactored the code so binlog_flush_cache handles both the trx-cache and stmt-cache. . There are functions that encapsulate the calls to binlog_flush_cache and make easier to read the code. . binlog_commit_flush_stmt_cache is now taking into account the result: success or error. sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-11-05 13:58:05 +00:00
-- die
}
BUG#56343 binlog_cache_use status is bigger than expected The binlog_cache_use is incremented twice when changes to a transactional table are committed, i.e. TC_LOG_BINLOG::log_xid calls is called. The problem happens because log_xid calls both binlog_flush_stmt_cache and binlog_flush_trx_cache without checking if such caches are empty thus unintentionally increasing the binlog_cache_use value twice. To fix the problem we avoided incrementing the binlog_cache_use if the cache is empty. We also decided to increment binlog_cache_use when the cache is truncated as the cache is used although its content is discarded and is not written to the binary log. Note that binlog_cache_use is incremented for both types of cache, transactional and non-transactional and that the behavior presented in this patch also applies to the binlog_cache_disk_use. Finally, we re-organized the code around the functions binlog_flush_trx_cache and binlog_flush_stmt_cache. mysql-test/extra/binlog_tests/binlog_cache_stat.test: Updated the test case with comments and additional tests to check both transactional and non-transactional changes and what happens when a is transaction either committed or aborted. mysql-test/suite/binlog/r/binlog_innodb.result: Updated the result file. mysql-test/suite/binlog/r/binlog_mixed_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_row_cache_stat.result: Updated the result file. mysql-test/suite/binlog/r/binlog_stm_cache_stat.result: Updated the result file. mysql-test/suite/binlog/t/binlog_mixed_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_row_cache_stat.test: Updated the test case with a new source file. mysql-test/suite/binlog/t/binlog_stm_cache_stat.test: Updated the test case with a new source file. sql/log_event.cc: Introduced debug information to check if Query_log_event("BEGIN"), Query_log_event("COMMIT") and Query_log_event("ROLLBACK"). sql/log_event.h: Guaranteed Xid_log_event is always classified as a transactional event.
2010-10-06 09:34:49 +01:00
drop table t1, t2;