The problem was that the maximum value of the transaction_prealloc_size
session system variable was ULONG_MAX which meant that it was possible
to cause the server to allocate excessive amounts of memory.
This patch fixes the problem by reducing the maxmimum value of
transaction_prealloc_size and transaction_alloc_block_size down
to 128K.
Note that transactions will still be able to allocate more than
128K if needed, this patch just reduces the amount that can be
preallocated - as well as the maximum size of the incremental
allocation blocks.
innodb_stats_sample_pages
Analysis: If you set the number of analyzed pages
to very low number compared to actual pages on
that table/index it randomly pics those pages
(default 8 pages), this leads to fact that query
after analyze table returns different results. If
the index tree is small, smaller than 10 *
n_sample_pages + total_external_size, then the
estimate is ok. For bigger index trees it is
common that we do not see any borders between
key values in the few pages we pick. But still
there may be n_sample_pages different key values,
or even more. And it just tries to
approximate to n_sample_pages (8).
Fix: (1) Introduced new dynamic configuration variable
innodb_stats_sample_traditional that retains
the current design. Default false.
(2) If traditional sample is not used we use
n_sample_pages = max(min(srv_stats_sample_pages,
index->stat_index_size),
log2(index->stat_index_size)*
srv_stats_sample_pages);
(3) Introduced new dynamic configuration variable
stat_modified_counter (default = 0) if set
sets lower bound for row updates when statistics is re-estimated.
If user has provided upper bound for how many rows needs to be updated
before we calculate new statistics we use minimum of provided value
and 1/16 of table every 16th round. If no upper bound is provided
(srv_stats_modified_counter = 0, default) then calculate new statistics
if 1 / 16 of table has been modified
since the last time a statistics batch was run.
We calculate statistics at most every 16th round, since we may have
a counter table which is very small and updated very often.
@param t table
@return true if the table has changed too much and stats need to be
recalculated
*/
#define DICT_TABLE_CHANGED_TOO_MUCH(t) \
((ib_int64_t) (t)->stat_modified_counter > (srv_stats_modified_counter ? \
ut_min(srv_stats_modified_counter, (16 + (t)->stat_n_rows / 16)) : \
16 + (t)->stat_n_rows / 16))
Normally, SET SESSION SQL_LOG_BIN is used by DBAs to run a
non-conflicting command locally only, ensuring it does not
get replicated.
Setting GLOBAL SQL_LOG_BIN would not require all sessions to
disconnect. When SQL_LOG_BIN is changed globally, it does not
immediately take effect for any sessions. It takes effect by
becoming the session-level default inherited at the start of
each new session, and this setting is kept and cached for the
duration of that session. Setting it intentionally is unlikely
to have a useful effect under any circumstance; setting it
unintentionally, such as while intending to use SET [SESSION]
is potentially disastrous. Accidentally using SET GLOBAL
SQL_LOG_BIN will not show an immediate effect to the user,
instead not having the desired session-level effect, and thus
causing other potential problems with local-only maintenance
being binlogged and executed on slaves; And transactions from
new sessions (after SQL_LOG_BIN is changed globally) are not
binlogged and replicated, which would result in irrecoverable
or difficult data loss.
This is the regular GLOBAL variables way to work, but in
replication context it does not look right on a working server
(with connected sessions) 'set global sql_log_bin' and none of
that connections is affected. Unexperienced DBA after noticing
that the command did "nothing" will change the session var and
most probably won't unset the global var, causing new sessions
to not be binlog.
Setting GLOBAL SQL_LOG_BIN allows DBA to stop binlogging on all
new sessions, which can be used to make a server "replication
read-only" without restarting the server. But this has such big
requirements, stop all existing connections, that it is more
likely to make a mess, it is too risky to allow the GLOBAL variable.
The statement 'SET GLOBAL SQL_LOG_BIN=N' will produce an error
in 5.5, 5.6 and 5.7. Reading the GLOBAL SQL_LOG_BIN will produce
a deprecation warning in 5.7.
Normally, SET SESSION SQL_LOG_BIN is used by DBAs to run a
non-conflicting command locally only, ensuring it does not
get replicated.
Setting GLOBAL SQL_LOG_BIN would not require all sessions to
disconnect. When SQL_LOG_BIN is changed globally, it does not
immediately take effect for any sessions. It takes effect by
becoming the session-level default inherited at the start of
each new session, and this setting is kept and cached for the
duration of that session. Setting it intentionally is unlikely
to have a useful effect under any circumstance; setting it
unintentionally, such as while intending to use SET [SESSION]
is potentially disastrous. Accidentally using SET GLOBAL
SQL_LOG_BIN will not show an immediate effect to the user,
instead not having the desired session-level effect, and thus
causing other potential problems with local-only maintenance
being binlogged and executed on slaves; And transactions from
new sessions (after SQL_LOG_BIN is changed globally) are not
binlogged and replicated, which would result in irrecoverable
or difficult data loss.
This is the regular GLOBAL variables way to work, but in
replication context it does not look right on a working server
(with connected sessions) 'set global sql_log_bin' and none of
that connections is affected. Unexperienced DBA after noticing
that the command did "nothing" will change the session var and
most probably won't unset the global var, causing new sessions
to not be binlog.
Setting GLOBAL SQL_LOG_BIN allows DBA to stop binlogging on all
new sessions, which can be used to make a server "replication
read-only" without restarting the server. But this has such big
requirements, stop all existing connections, that it is more
likely to make a mess, it is too risky to allow the GLOBAL variable.
The statement 'SET GLOBAL SQL_LOG_BIN=N' will produce an error
in 5.5, 5.6 and 5.7. Reading the GLOBAL SQL_LOG_BIN will produce
a deprecation warning in 5.7.
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
ACCEPTED BUT PARSED INCORRECTLY
When we are setting the value in a system variable,
We can set it like
set sys_var="Iden1.Iden2"; //1
set sys_var='Iden1.Iden2'; //2
set sys_var=Iden1.Iden2; //3
set sys_var=.ident1.ident2; //4
set sys_var=`Iden1.Iden2`; //5
While parsing, for case 1(when ANSI_QUOTES is enable) and 2,
we will take as string literal(we will make item of type Item_string).
for case 3 & 4, taken as Item_field, where Iden1 is a table name and
iden2 is a field name.
for case 5, again Item_field type, where iden1.iden2 is taken as
field name.
Now in case 1, when we are assigning some value to system variable
(which can take string or enumerate type data), we are setting only
field part.
This means only iden2 value will be set for system variable. This
result in wrong result.
Solution:
(for string type) We need to Document that we are not allowed to set
system variable which takes string as identifier, otherwise result
in unexpected behaviour.
(for enumerate type)
if we pass iden1.iden2, we will give an error ER_WRONG_TYPE_FOR_VAR
(Incorrect argument type to variable).
mysql-test/suite/sys_vars/t/general_log_file_basic.test:
Earlier we used to give ER_WRONG_VALUE_FOR_VAR error, but in the patch of
(Bug32748-Inconsistent handling of assignments to general_log_file/slow_query_log_file)
they quoted this line.But i am not able to find any relation of this with the changes of
patch. So i think We should give error in this case.
mysql-test/suite/sys_vars/t/slow_query_log_file_basic.test:
Earlier we used to give ER_WRONG_VALUE_FOR_VAR error, but in the patch of
(Bug32748-Inconsistent handling of assignments to general_log_file/slow_query_log_file)
they quoted this line.But i am not able to find any relation of this with the changes of
patch. So i think We should give error in this case.
ACCEPTED BUT PARSED INCORRECTLY
When we are setting the value in a system variable,
We can set it like
set sys_var="Iden1.Iden2"; //1
set sys_var='Iden1.Iden2'; //2
set sys_var=Iden1.Iden2; //3
set sys_var=.ident1.ident2; //4
set sys_var=`Iden1.Iden2`; //5
While parsing, for case 1(when ANSI_QUOTES is enable) and 2,
we will take as string literal(we will make item of type Item_string).
for case 3 & 4, taken as Item_field, where Iden1 is a table name and
iden2 is a field name.
for case 5, again Item_field type, where iden1.iden2 is taken as
field name.
Now in case 1, when we are assigning some value to system variable
(which can take string or enumerate type data), we are setting only
field part.
This means only iden2 value will be set for system variable. This
result in wrong result.
Solution:
(for string type) We need to Document that we are not allowed to set
system variable which takes string as identifier, otherwise result
in unexpected behaviour.
(for enumerate type)
if we pass iden1.iden2, we will give an error ER_WRONG_TYPE_FOR_VAR
(Incorrect argument type to variable).
Added variable "OLD_MODE" that can be used to turn off the new behavior
mysql-test/r/insert.result:
Added test case
mysql-test/r/mysqld--help.result:
Added old_mode
mysql-test/suite/sys_vars/r/old_mode_basic.result:
Added testing of new variable
mysql-test/suite/sys_vars/t/old_mode_basic.test:
Added testing of new variable
mysql-test/t/insert.test:
Added test case
sql/sql_class.h:
Added bit flags for OLD_MODE
sql/sql_insert.cc:
Disable duplicate key warnings for INSERT IGNORE of OLD_MODE NO_DUP_KEY_WARNINGS_WITH_IGNORE is used
sql/sql_show.cc:
Don't show progress reporting on SHOW PROCESSLIST if OLD_MODE NO_PROGRESS_INFO is used
sql/sys_vars.cc:
Added OLD_MODE
The maximum value for innodb_thread_sleep_delay is 4294967295 (32-bit) or
18446744073709551615 (64-bit) microseconds. This is way too big, since
the max value of innodb_thread_sleep_delay is limited by
innodb_adaptive_max_sleep_delay if that value is set to non-zero value
(its default is 150,000).
Solution
The maximum value of innodb_thread_sleep_delay should be the same as
the maximum value of innodb_adaptive_max_sleep_delay, which is 1000000.
Approved by Jimmy, rb#4429
The maximum value for innodb_thread_sleep_delay is 4294967295 (32-bit) or
18446744073709551615 (64-bit) microseconds. This is way too big, since
the max value of innodb_thread_sleep_delay is limited by
innodb_adaptive_max_sleep_delay if that value is set to non-zero value
(its default is 150,000).
Solution
The maximum value of innodb_thread_sleep_delay should be the same as
the maximum value of innodb_adaptive_max_sleep_delay, which is 1000000.
Approved by Jimmy, rb#4429
- thread_pool_size command line option upper limit increased to 100 000
(same as for max_connections)
- thread_pool_size system variable upper limit is maximum of 128 or
the value given at command line
- thread groups are now allocated dynamically
Different limit for command line option and system variable was done to
avoid additional mutex for all_groups and threadpool_max_size.
Problem:
sys_vars.rpl_init_slave_func test was failing sporadically
on 5.5+.
Fix:
Added assert condition after wait for checks.
Recorded test and enabled it.
Problem:
sys_vars.rpl_init_slave_func test was failing sporadically
on 5.5+.
Fix:
Added assert condition after wait for checks.
Recorded test and enabled it.
BUG#12535301- SYS_VARS.RPL_INIT_SLAVE_FUNC MISMATCHES IN DAILY-5.5
Problem:
sys_vars.rpl_init_slave_func test was not recorded after
the last edit. It was disabled on 5.1 after seeing failures
due to the above reason.
No old failures as this suite never ran with pb2 on 5.1
Fix:
Added assert condition after wait for checks.
Recorded test and enabled it.
BUG#12535301- SYS_VARS.RPL_INIT_SLAVE_FUNC MISMATCHES IN DAILY-5.5
Problem:
sys_vars.rpl_init_slave_func test was not recorded after
the last edit. It was disabled on 5.1 after seeing failures
due to the above reason.
No old failures as this suite never ran with pb2 on 5.1
Fix:
Added assert condition after wait for checks.
Recorded test and enabled it.
-Added test and extra code to ensure we don't leave keyread on for a handler table.
-Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
mysql-test/include/default_mysqld.cnf:
Run test suite with smaller aria keybuffer size
mysql-test/suite/maria/maria3.result:
Run test suite with smaller aria keybuffer size
mysql-test/suite/sys_vars/r/aria_pagecache_buffer_size_basic.result:
Run test suite with smaller aria keybuffer size
sql/handler.cc:
Disable key read (extra safety if something went wrong)
sql/multi_range_read.cc:
Ensure we have don't leave keyread on for secondary_file
sql/opt_range.cc:
Simplify code with mark_columns_used_by_index_no_reset()
Ensure that read_keys_and_merge() disableds keyread if it enables it
sql/opt_subselect.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_derived.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_select.cc:
Use 'enable_keyread()' instead of calling HA_EXTRA_RESET. (Makes debugging easier)
Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
Remove not anymore used argument for create_internal_tmp_table()
More DBUG
sql/sql_select.h:
Remove not anymore used argument for create_internal_tmp_table()
Change default for innodb_use_fallocate to FALSE, due to bugs in older Linux kernels (posix_fallocate() does not always guarantee that file size is like one specified)
This fixes that by default LOAD DATA INFILE will not generate the error:
"Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage..."
mysql-test/suite/sys_vars/r/max_binlog_cache_size_basic.result:
Updated test case
mysql-test/suite/sys_vars/r/max_binlog_stmt_cache_size_basic.result:
Updated test case
sql/sys_vars.cc:
Increase default value of max_binlog_cache_size and max_binlog_stmt_cache_size to ulonglong_max.
buf_page_get_gen(): Do not attempt to decompress a compressed-only
page when mode == BUF_PEEK_IF_IN_POOL. This mode is only being used by
btr_search_drop_page_hash_when_freed(). There cannot be any adaptive
hash index pointing to a page that does not exist in uncompressed
format in the buffer pool.
innodb_buffer_pool_evict_update(): New function for debug builds, to handle
SET GLOBAL innodb_buffer_pool_evicted='uncompressed'
by evicting all uncompressed page frames of compressed tablespaces
from the buffer pool.
rb#1873 approved by Jimmy Yang
buf_page_get_gen(): Do not attempt to decompress a compressed-only
page when mode == BUF_PEEK_IF_IN_POOL. This mode is only being used by
btr_search_drop_page_hash_when_freed(). There cannot be any adaptive
hash index pointing to a page that does not exist in uncompressed
format in the buffer pool.
innodb_buffer_pool_evict_update(): New function for debug builds, to handle
SET GLOBAL innodb_buffer_pool_evicted='uncompressed'
by evicting all uncompressed page frames of compressed tablespaces
from the buffer pool.
rb#1873 approved by Jimmy Yang
