tree for embedded server
Test case for bug #56251 "Deadlock with INSERT
DELAYED and MERGE tables" can't be run against
embedded server. Embedded server converts all
DELAYED INSERTs into ordinary INSERTs and this
test can't work properly if such conversion
happens.
Moved this test from merge.test to delayed.test
which is skipped if test suite is run with
--embedded-server option.
Subselect executes twice, at JOIN::optimize stage
and at JOIN::execute stage. At optimize stage
Innodb prebuilt struct which is used for the
retrieval of column values is initialized in.
ha_innobase::index_read(), prebuilt->sql_stat_start is true.
After QUICK_ROR_INTERSECT_SELECT finished his job it
restores read_set/write_set bitmaps with initial values
and deactivates one of the handlers used by
QUICK_ROR_INTERSECT_SELECT in JOIN::cleanup
(it's the case when we reuse original handler as one of
handlers required by QUICK_ROR_INTERSECT_SELECT object).
On second subselect execution inactive handler is activated
in QUICK_RANGE_SELECT::reset, file->ha_index_init().
In ha_index_init Innodb prebuilt struct is reinitialized
with inappropriate read_set/write_set bitmaps. Further
reinitialization in ha_innobase::index_read() does not
happen as prebuilt->sql_stat_start is false.
It leads to partial retrieval of required field values
and we get a mix of field values from different records
in the record buffer.
The fix is to reset
read_set/write_set bitmaps as these values
are required for proper intialization of
internal InnoDB struct which is used for
the retrieval of column values
(see build_template(), ha_innodb.cc)
mysql-test/include/index_merge_ror_cpk.inc:
test case
mysql-test/r/index_merge_innodb.result:
test case
mysql-test/r/index_merge_myisam.result:
test case
sql/opt_range.cc:
if ROR merge scan is used we need to reset
read_set/write_set bitmaps as these values
are required for proper intialization of
internal InnoDB struct which is used for
the retrieval of column values
(see build_template(), ha_innodb.cc)
adding new indexes
A fast alter table requires that the existing (old) table
and indices are unchanged (i.e only new indices can be
added). To verify this, the layout and flags of the old
table/indices are compared for equality with the new.
The PACK_KEYS option is a no-op in InnoDB, but the flag
exists, and is used in the table compare. We need to
check this (table) option flag before deciding whether an
index should be packed or not. If the table has
explicitly set PACK_KEYS to 0, the created indices should
not be marked as packed/packable.
compression protocol.
The loss of connection was caused by a malformed packet
sent by the server in case when query cache was in use.
When storing data in the query cache, the query cache
memory allocation algorithm had a tendency to reduce
the amount of memory block necessary to store a result
set, up to finally storing the entire result set in a single
block. With a significant result set, this memory block
could turn out to be quite large - 30, 40 MB and on.
When such a result set was sent to the client, the entire
memory block was compressed and written to network as a
single network packet. However, the length of the
network packet is limited by 0xFFFFFF (16MB), since
the packet format only allows 3 bytes for packet length.
As a result, a malformed, overly large packet
with truncated length would be sent to the client
and break the client/server protocol.
The solution is, when sending result sets from the query
cache, to ensure that the data is chopped into
network packets of size <= 16MB, so that there
is no corruption of packet length. This solution,
however, has a shortcoming: since the result set
is still stored in the query cache as a single block,
at the time of sending, we've lost boundaries of individual
logical packets (one logical packet = one row of the result
set) and thus can end up sending a truncated logical
packet in a compressed network packet.
As a result, on the client we may require more memory than
max_allowed_packet to keep, both, the truncated
last logical packet, and the compressed next packet.
This never (or in practice never) happens without compression,
since without compression it's very unlikely that
a) a truncated logical packet would remain on the client
when it's time to read the next packet
b) a subsequent logical packet that is being read would be
so large that size-of-new-packet + size-of-old-packet-tail >
max_allowed_packet.
To remedy this issue, we send data in 1MB sized packets,
that's below the current client default of 16MB for
max_allowed_packet, but large enough to ensure there is no
unnecessary overhead from too many syscalls per result set.
sql/net_serv.cc:
net_realloc() modified: consider already used memory
when compare packet buffer length
sql/sql_cache.cc:
modified Query_cache::send_result_to_client: send result to client
in chunks limited by 1 megabyte.
The problem was that RENAME TABLE caused an assert if the system variable
lower_case_table_names was 2 (default on Mac OS X) and the old table name
was given in upper case. This caused lowercase_table2.test to fail.
The assert checks that an exclusive metadata lock is held by the connection
trying to do RENAME TABLE - specificially during updates of table triggers.
The assert was triggered since the check is case sensitive and the lock
was held on the normalized (lower case) version of the table name.
This patch fixes the problem by making sure a normalized version of the
table name is used for the metadata lock check, while using a non-normalized
version of the table name for the rename of trigger files. The same is done
for ALTER TABLE ... RENAME.
Regression testing for the bug itself is already covered by
lowercase_table2.test. Additional coverage added to lowercase_fs_off.test.
Before this fix, the server could crash inside a memcpy when reading data
from the EVENTS_WAITS_CURRENT / HISTORY / HISTORY_LONG tables.
The root cause is that the length used in a memcpy could be corrupted,
when another thread writes data in the wait record being read.
Reading unsafe data is ok, per design choice, and the code does sanitize
the data in general, but did not sanitize the length given to memcpy.
The fix is to also sanitize the schema name / object name / file name
length when extracting the data to produce a row.
Fix compiler warning:
trx/trx0undo.c: In function 'trx_undo_truncate_end':
trx/trx0undo.c:1069:14: error: variable 'rseg' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
trx/trx0undo.c: In function 'trx_undo_set_state_at_prepare':
trx/trx0undo.c:1871:16: error: variable 'page_hdr' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
trx/trx0trx.c: In function 'trx_prepare_off_kernel':
trx/trx0trx.c:1808:11: error: variable 'update_hdr_page' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
trx/trx0sys.c: In function 'trx_sys_create_doublewrite_buf':
trx/trx0sys.c:244:15: error: variable 'new_block' set but not used [-Werror=unused-but-set-variable]
Fix compiler warnings:
trx/trx0roll.c: In function 'trx_undo_arr_remove_info':
trx/trx0roll.c:744:9: error: variable 'n' set but not used [-Werror=unused-but-set-variable]
trx/trx0roll.c:743:9: error: variable 'n_used' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
trx/trx0purge.c: In function 'trx_purge_rec_release':
trx/trx0purge.c:1071:18: error: variable 'arr' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
trx/trx0purge.c: In function 'trx_purge_rseg_get_next_history_log':
trx/trx0purge.c:660:15: error: variable 'seg_hdr' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
trx/trx0purge.c: In function 'trx_purge_add_update_undo_to_history':
trx/trx0purge.c:307:15: error: variable 'seg_header' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
trx/trx0purge.c: In function 'trx_purge_add_update_undo_to_history':
trx/trx0purge.c:309:16: error: variable 'page_header' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
row/row0vers.c: In function 'row_vers_impl_x_locked_off_kernel':
row/row0vers.c:74:9: error: variable 'err' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
row/row0umod.c: In function 'row_undo_mod_clust_low':
row/row0umod.c:117:9: error: variable 'success' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
row/row0purge.c: In function 'row_purge_step':
row/row0purge.c:687:9: error: variable 'err' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
que/que0que.c: In function 'que_run_threads_low':
que/que0que.c:1287:9: error: variable 'cumul_resource' set but not used [-Werror=unused-but-set-variable]
Fix compiler warnings:
os/os0file.c: In function 'os_file_create':
os/os0file.c:1371:14: error: variable 'purpose_str' set but not used [-Werror=unused-but-set-variable]
os/os0file.c:1370:14: error: variable 'type_str' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
log/log0recv.c: In function 'recv_synchronize_groups':
log/log0recv.c:562:14: error: variable 'limit_lsn' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
log/log0recv.c: In function 'recv_recovery_from_checkpoint_start_func':
log/log0recv.c:2894:14: error: variable 'archived_lsn' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
handler/i_s.cc: In function 'int trx_i_s_common_fill_table(THD*, TABLE_LIST*, COND*)':
handler/i_s.cc:931:8: error: variable 'ret' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
handler/ha_innodb.cc: In function 'void innobase_drop_database(handlerton*, char*)':
handler/ha_innodb.cc:7010:6: error: variable 'error' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
handler/ha_innodb.cc: In function 'bool innodb_show_status(handlerton*, THD*, bool (*)(THD*, const char*, uint, const char*, uint, const char*, uint))':
handler/ha_innodb.cc:8851:7: error: variable 'result' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
eval/eval0eval.c: In function 'eval_notfound':
eval/eval0eval.c:388:14: error: variable 'arg2' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
dict/dict0dict.c: In function 'dict_index_print_low':
dict/dict0dict.c:4444:14: error: variable 'type_string' set but not used [-Werror=unused-but-set-variable]
tables".
Attempting to issue an INSERT DELAYED statement for a MERGE
table might have caused a deadlock if it happened as part of
a transaction or under LOCK TABLES, and there was a concurrent
DDL or LOCK TABLES ... WRITE statement which tried to lock one
of its underlying tables.
The problem occurred when a delayed insert handler thread tried
to open a MERGE table and discovered that to do this it had also
to open all underlying tables and hence acquire metadata
locks on them. Since metadata locks on the underlying tables were
not pre-acquired by the connection thread executing INSERT DELAYED,
attempts to do so might lead to waiting. In this case the
connection thread had to wait for the delayed insert thread.
If the thread which was preventing the lock on the underlying table
from being acquired had to wait for the connection thread (due to
this or other metadata locks), a deadlock occurred.
This deadlock was not detected by the MDL deadlock detector since
waiting for the handler thread by the connection thread is not
represented in the wait-for graph.
This patch solves the problem by ensuring that the delayed
insert handler thread never tries to open underlying tables
of a MERGE table. Instead open_tables() is aborted right after
the parent table is opened and a ER_DELAYED_NOT_SUPPORTED
error is emitted (which is passed to the connection thread and
ultimately to the user).
mysql-test/r/merge.result:
Added test for bug #56251 "Deadlock with INSERT DELAYED and
MERGE tables".
mysql-test/t/merge.test:
Added test for bug #56251 "Deadlock with INSERT DELAYED and
MERGE tables".
sql/sql_base.cc:
Changed open_n_lock_single_table() to take prelocking strategy
as an argument instead of always using DML_prelocking_strategy.
sql/sql_base.h:
Changed open_n_lock_single_table() to take prelocking strategy
as an argument instead of always using DML_prelocking_strategy.
Added a version of this function which is compatible with old
signature.
sql/sql_insert.cc:
When opening MERGE table in delayed insert thread stop and emit
ER_DELAYED_NOT_SUPPORTED right after opening main table and
before opening underlying tables. This ensures that we won't
try to acquire metadata lock on underlying tables which might
lead to a deadlock.
This is achieved by using special prelocking strategy which
abort open_tables() process as soon as we discover that we
have opened table with engine which doesn't support delayed
inserts.
The crash during boot was caused by a DBUG_PRINT statement in fill_schema_schemata() (in
sql_show.cc). This DBUG_PRINT statement contained several instances of %s in the format
string and for one of these we gave a NULL pointer as the argument. This caused the
call to vsnprintf() to crash when running on Solaris.
The fix for this problem is to replace the call to vsnprintf() with my_vsnprintf()
which handles that a NULL pointer is passed as argumens for %s.
This patch also extends my_vsnprintf() to support %i in the format string.
dbug/dbug.c:
Replace the use of vsnprintf() with my_vsnprintf(). On some platforms
vsnprintf() did not handle that a NULL pointer was given as an argument
for a %s in the format string.
include/mysql/service_my_snprintf.h:
Add support for %i in format string to my_vsnprintf().
strings/my_vsnprintf.c:
Add support for %i in format string to my_vsnprintf().
unittest/mysys/my_vsnprintf-t.c:
Add unit tests for %i in format string to my_vsnprintf().
Fix compiler warning:
dict/dict0crea.c: In function 'dict_create_index_tree_step':
dict/dict0crea.c:630:16: error: variable 'table' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
buf/buf0flu.c: In function 'buf_flush_batch':
buf/buf0flu.c:1274:9: error: variable 'old_page_count' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
buf/buf0flu.c: In function 'buf_flush_delete_from_flush_rbt':
buf/buf0flu.c:131:8: error: variable 'ret' set but not used [-Werror=unused-but-set-variable]
Bug#56657: Test still uses "--exec rm -f ..." which is non-portable
Bug#56601: Test uses Unix path for temporary file, fails, and writes misleading message
Several tests that was written in a non portable way (failed on windows)
Fixed by
1) backporting the fix for replace_result to also apply to list_files
(mysqltest from mysql-trunk)
2) replacing all #p#/#sp#/#tmp# to #P#/#SP#/#TMP#/
(innodb always converts filenames to lower case in windows).
3) replacing '--exec rm -f' with '--remove_files_wildcard'
4) replacing a perl snippet with '--write_file'
client/mysqltest.cc:
backport from mysql-trunk to allow replace_result to apply
also on list_files
mysql-test/suite/parts/inc/partition_check_drop.inc:
Compensate for differences between innodb on windows vs unix.
Using mysqltest command, instead of unix command to remove files.
mysql-test/suite/parts/inc/partition_crash.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_fail.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_layout.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_layout_check1.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_layout_check2.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/r/partition_recover_myisam.result:
updated result
mysql-test/suite/parts/r/partition_special_myisam.result:
updated result
mysql-test/suite/parts/t/part_supported_sql_func_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter1_1_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter1_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter2_1_1_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter2_1_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter2_2_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter4_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_debug_sync_innodb.test:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/t/partition_recover_myisam.test:
more generic suppression (failed in windows)
mysql-test/suite/parts/t/partition_special_myisam.test:
Using portable mysqltest command 'write_file' instead of perl snippet.
Fix compiler warning:
btr/btr0sea.c: In function 'btr_search_update_hash_on_delete':
btr/btr0sea.c:1498:9: error: variable 'found' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
btr/btr0pcur.c: In function 'btr_pcur_move_backward_from_page':
btr/btr0pcur.c:455:9: error: variable 'space' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
btr/btr0cur.c: In function 'btr_free_externally_stored_field':
btr/btr0cur.c:4281:16: error: variable 'rec_block' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
btr/btr0cur.c: In function 'btr_cur_optimistic_update':
btr/btr0cur.c:1839:10: error: variable 'orig_rec' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
btr/btr0btr.c: In function 'btr_compress':
btr/btr0btr.c:2564:9: error: variable 'level' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
btr/btr0btr.c: In function 'btr_page_split_and_insert':
btr/btr0btr.c:1898:11: error: variable 'insert_page' set but not used [-Werror=unused-but-set-variable]
Fix compiler warning:
./include/ut0rnd.ic: In function 'ut_rnd_gen_ulint':
./include/ut0rnd.ic:88:8: error: variable 'n_bits' set but not used [-Werror=unused-but-set-variable]
