Clarified error messages related to unsafe statements:
- avoid the internal technical term "row injection"
- use 'binary log' instead of 'binlog'
- avoid the word 'unsafeness'
Stored routine DDL statements use statement-based replication
regardless of the current binlog format. The problem here was
that if a DDL statement failed during metadata lock acquisition
or opening of mysql.proc, the binlog format would not be reset
before returning. So the following DDL or DML statements are
binlogged with a wrong binlog format, which causes the slave
to stop.
The problem can be resolved by grabbing an exclusive MDL lock firstly
instead of clearing the current binlog format. So that the binlog
format will not be affected when the lock grab returns directly with
an error. The same way is taken to open a proc table for update.
lock monitor thread, it may have locks that are granted to waited to
waiting transactions. These waiting transactions will need to be woken
up but their trx->lock_wait_timeout flag will be FALSE causing the old
code to break. What we need is a flag that covers the entire lock
release process not individual transactions. The fix is to move the
flag out of trx_t and into srv_sys_t.
parallel mode
The failure has nothing to do with parallel, but rather on the
order the tests are executed. In this case, the test
binlog_tmp_table (lets call it test2) was not ensuring that the
binary logs would be reset when it started. Later the test issues
a mysqlbinlog .../master-bin.000002 | mysql ... If the test that
was executed before this one (lets call it test1) had issued a
flush logs, then the file in use in test1 (master-bin.000002)
would not actually match the one that was expected. Eventually,
this would cause the statements logged in test1 to be replayed,
instead of the ones logged in the beginning of test2.
We fix this by:
1. adding RESET MASTER to the beginning of binlog_tmp_table
2. setting dynamically the file to use in binlog_tmp_table
Only #1 was needed, but the two make the tests cases more robust.
The problem is that message resource (message.rc) is compiled as part of static library
sql.lib rather than with executable mysqld.exe. resource files do not work in static
libraries.
The fix is to add message.rc to mysqld.exe source files list.
was skipped because another flush batch was active. This is to
ensure that the when we return success then it is guaranteed that
all pages up to the lsn_limit have been flushed to the disk.
Statements with CONNECTION_ID were forced to be kept in the transactional
cache and by consequence non-transactional changes that were supposed to
be flushed ahead of the transaction were kept in the transactional cache.
This happened because after BUG#51894 any statement whose thd's
thread_specific_used was set was kept in the transactional cache. The idea
was to keep changes on temporary tables in the transactional cache. However,
the thread_specific_used was set not only for statements that accessed
temporary tables but also when the CONNECTION_ID was used.
To fix the problem, we created a new variable to keep track of updates
to temporary tables.
The bug was a side effect of WL#5030 (fix header files) and
WL#5161 (CMake).
The problem was that CMake-generated config.h (and my_config.h
as a copy of it) had a header guard. GNU autotools-generated
[my_]config.h did not. During WL#5030 the order of header files
was changed, so the following started to happen (using GNU autotools,
in embedded server):
- my_config.h included, defining HAVE_OPENSSL
- my_global.h included, un-defining HAVE_OPENSSL
- zlib.h included, including config.h,
defining HAVE_OPENSSL again.
The fix is to check HAVE_OPENSSL in conjuction with EMBEDDED_LIBRARY.
More common fix would be to define a macros as HAVE_OPENSSL && !EMBEDDED_LIBRARY
and use it instead of HAVE_OPENSSL.
------------------------------------------------------------
revno: 3400
revision-id: mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao
parent: vasil.dimov@oracle.com-20100414200413-ktulgshvdp0smxk0
committer: Marko Makela <mmakela@bk-internal.mysql.com>
branch nick: mysql-5.1-innodb
timestamp: Thu 2010-04-15 09:01:22 +0200
message:
storage/innodb_plugin: Relax too strict assertions about prefix
indexed BLOBs for ROW_FORMAT=DYNAMIC and ROW_FORMAT_COMPRESSED tables
(Bug #52746). In these tables, the locally stored prefix of a BLOB can
be as small as 20 bytes (BTR_EXTERN_FIELD_REF_SIZE). ROW_FORMAT=REDUNDANT
and ROW_FORMAT=COMPACT store a prefix of 768 bytes (REC_MAX_INDEX_COL_LEN).
trx_undo_rec_get_col_val(): Relax the ut_ad() assertion and add a
reference to dtuple_convert_big_rec().
trx_undo_rec_get_partial_row(): Relax the ut_a() assertion that
prompted Bug #52746.
modified:
storage/innodb_plugin/ChangeLog 2425@16c675df-0fcb-4bc9-8058-dcc011a37293:branches%2Fzip%2FChangeLog
storage/innodb_plugin/trx/trx0rec.c 2@16c675df-0fcb-4bc9-8058-dcc011a37293:trunk%2Ftrx%2Ftrx0rec.c
The bug was a side effect of WL#5030 (fix header files) and
WL#5161 (CMake).
The problem was that CMake-generated config.h (and my_config.h
as a copy of it) had a header guard. GNU autotools-generated
[my_]config.h did not. During WL#5030 the order of header files
was changed, so the following started to happen (using GNU autotools,
in embedded server):
- my_config.h included, defining HAVE_OPENSSL
- my_global.h included, un-defining HAVE_OPENSSL
- zlib.h included, including config.h,
defining HAVE_OPENSSL again.
The fix is to change the order of header file, moving zlib.h
to the top of the header list. More proper fix would be to wrap
unguarded auto-generated [my_]config.h by guarded non-generated
header file.
Splitting the kernel mutex at once is a complex task, several mutexes will
be factored out of it. This is one of the easier ones. This mutex sits below
the kernel mutex and is used by the threads infra-structure. It is important
for the multi-threaded purge because for multi-threaded purge we will need
to activate and use InnoDB task queue and query thread scheduling code.
rb://285
transaction
BUG#52616 Temp table prevents switch binlog format from STATEMENT to ROW
Before the WL#2687 and BUG#46364, every non-transactional change that happened
after a transactional change was written to trx-cache and flushed upon
committing the transaction. WL#2687 and BUG#46364 changed this behavior and
non-transactional changes are now written to the binary log upon committing
the statement.
A binary log event is identified as transactional or non-transactional through
a flag in the Log_event which is set taking into account the underlie storage
engine on what it is stems from. In the current bug, this flag was not being
set properly when the DROP TEMPORARY TABLE was executed.
However, while fixing this bug we figured out that changes to temporary tables
should be always written to the trx-cache if there is an on-going transaction.
Otherwise, binlog events in the reversed order would be produced.
Regarding concurrency, keeping changes to temporary tables in the trx-cache is
also safe as temporary tables are only visible to the owner connection.
In this patch, we classify the following statements as unsafe:
1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp
2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam
3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam
On the other hand, the following statements are classified as safe:
1 - INSERT INTO t_innodb SELECT * FROM t_myisam_temp
2 - INSERT INTO t_myisam_temp SELECT * FROM t_innodb
The patch also guarantees that transactions that have a DROP TEMPORARY are
always written to the binary log regardless of the mode and the outcome:
commit or rollback. In particular, the DROP TEMPORARY is extended with the
IF EXISTS clause when the current statement logging format is set to row.
Finally, the patch allows to switch from STATEMENT to MIXED/ROW when there
are temporary tables but the contrary is not possible.
newly introduced metadata locks.
Previously the behavior was deterministic and if several LOCKs were
waiting the first one of them was released by UNLOCK (in chronological
order).
Now (with MDLs) the behavior is undefined and since we do not know in
what order to --reap the connections we simply disconnect them without
reaping.
Merge r6915:6992 from branches/innodb+multipbp (i.e.: all the changes
made since it's creation)
This also reverts r6930 to branches/innodb+ because a different
solution for that issue is already present in innodb+multibp which
is being merged.
After this commit branches/innodb+multibp should be discarded
and this branch should become our main development tree.
------------------------------------------------------------------------
r6915 | sbains | 2010-03-31 07:33:43 +0300 (Wed, 31 Mar 2010) | 1 line
Changed paths:
A /branches/innodb+multibp (from /branches/innodb+:6914)
Creating a branch for the multiple buffer pool
------------------------------------------------------------------------
r6916 | sbains | 2010-03-31 08:21:00 +0300 (Wed, 31 Mar 2010) | 3 lines
Changed paths:
M /branches/innodb+multibp/CMakeLists.txt
M /branches/innodb+multibp/btr/btr0btr.c
M /branches/innodb+multibp/btr/btr0cur.c
M /branches/innodb+multibp/btr/btr0sea.c
M /branches/innodb+multibp/buf/buf0buddy.c
M /branches/innodb+multibp/buf/buf0buf.c
M /branches/innodb+multibp/buf/buf0flu.c
M /branches/innodb+multibp/buf/buf0lru.c
M /branches/innodb+multibp/buf/buf0rea.c
M /branches/innodb+multibp/handler/ha_innodb.cc
M /branches/innodb+multibp/handler/i_s.cc
M /branches/innodb+multibp/ibuf/ibuf0ibuf.c
M /branches/innodb+multibp/include/buf0buddy.h
M /branches/innodb+multibp/include/buf0buddy.ic
M /branches/innodb+multibp/include/buf0buf.h
M /branches/innodb+multibp/include/buf0buf.ic
M /branches/innodb+multibp/include/buf0flu.h
M /branches/innodb+multibp/include/buf0flu.ic
M /branches/innodb+multibp/include/buf0lru.h
M /branches/innodb+multibp/include/buf0rea.h
M /branches/innodb+multibp/include/buf0types.h
M /branches/innodb+multibp/include/ibuf0ibuf.ic
M /branches/innodb+multibp/include/srv0srv.h
M /branches/innodb+multibp/include/univ.i
M /branches/innodb+multibp/log/log0log.c
M /branches/innodb+multibp/log/log0recv.c
M /branches/innodb+multibp/mem/mem0mem.c
M /branches/innodb+multibp/page/page0zip.c
M /branches/innodb+multibp/srv/srv0srv.c
M /branches/innodb+multibp/srv/srv0start.c
M /branches/innodb+multibp/trx/trx0trx.c
M /branches/innodb+multibp/trx/trx0undo.c
branches/innodb+multibp: Unable to crash it with UNIV_DEBUG and UNIV_SYNC_DEBUG
with both ibtests and Sysbench. The patch now needs a workout from Michael.
------------------------------------------------------------------------
r6917 | sbains | 2010-03-31 08:56:18 +0300 (Wed, 31 Mar 2010) | 2 lines
Changed paths:
M /branches/innodb+multibp/handler/ha_innodb.cc
branches/innodb+multibp: Fix error introduced in r6916.
------------------------------------------------------------------------
r6923 | sbains | 2010-03-31 15:16:04 +0300 (Wed, 31 Mar 2010) | 3 lines
Changed paths:
M /branches/innodb+multibp/btr/btr0cur.c
M /branches/innodb+multibp/buf/buf0buddy.c
M /branches/innodb+multibp/buf/buf0buf.c
M /branches/innodb+multibp/buf/buf0flu.c
M /branches/innodb+multibp/buf/buf0lru.c
M /branches/innodb+multibp/include/buf0buddy.ic
M /branches/innodb+multibp/include/buf0buf.h
M /branches/innodb+multibp/include/buf0buf.ic
M /branches/innodb+multibp/include/buf0flu.ic
M /branches/innodb+multibp/page/page0zip.c
branches/innodb+multibp: Fix whitespace issues. Add function
buf_pool_from_block(). Add some comments to parameters.
------------------------------------------------------------------------
r6932 | sbains | 2010-04-01 01:12:07 +0300 (Thu, 01 Apr 2010) | 4 lines
Changed paths:
M /branches/innodb+multibp/include/buf0buf.ic
M /branches/innodb+multibp/include/univ.i
branches/innodb+multibp: Remove bogus assertion. It's possible for the space
and offset of a page to be undefined during the lifecycle of a page. Remove
the debug #defines from univ.i.
------------------------------------------------------------------------
r6933 | sbains | 2010-04-01 01:22:40 +0300 (Thu, 01 Apr 2010) | 2 lines
Changed paths:
M /branches/innodb+multibp/srv/srv0start.c
branches/innodb+multibp: Fix whitespace issues.
------------------------------------------------------------------------
r6934 | sbains | 2010-04-01 01:53:18 +0300 (Thu, 01 Apr 2010) | 2 lines
Changed paths:
M /branches/innodb+multibp/CMakeLists.txt
M /branches/innodb+multibp/ChangeLog
M /branches/innodb+multibp/buf/buf0buf.c
M /branches/innodb+multibp/buf/buf0flu.c
M /branches/innodb+multibp/handler/ha_innodb.cc
M /branches/innodb+multibp/include/buf0buf.h
M /branches/innodb+multibp/include/buf0buf.ic
M /branches/innodb+multibp/include/buf0flu.ic
M /branches/innodb+multibp/include/srv0srv.h
M /branches/innodb+multibp/include/sync0sync.h
M /branches/innodb+multibp/include/trx0purge.h
M /branches/innodb+multibp/include/ut0ut.h
M /branches/innodb+multibp/include/ut0ut.ic
M /branches/innodb+multibp/lock/lock0lock.c
M /branches/innodb+multibp/log/log0recv.c
M /branches/innodb+multibp/mtr/mtr0mtr.c
M /branches/innodb+multibp/mysql-test/innodb_bug38231.test
A /branches/innodb+multibp/mysql-test/innodb_bug51920.result (from /branches/innodb+/mysql-test/innodb_bug51920.result:6931)
A /branches/innodb+multibp/mysql-test/innodb_bug51920.test (from /branches/innodb+/mysql-test/innodb_bug51920.test:6931)
M /branches/innodb+multibp/row/row0sel.c
M /branches/innodb+multibp/srv/srv0srv.c
M /branches/innodb+multibp/srv/srv0start.c
M /branches/innodb+multibp/sync/sync0sync.c
M /branches/innodb+multibp/trx/trx0purge.c
branches/innodb+multibp: Merge revisions r6914:6931 from branches/innodb+
------------------------------------------------------------------------
r6935 | sbains | 2010-04-01 02:08:32 +0300 (Thu, 01 Apr 2010) | 3 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0flu.c
M /branches/innodb+multibp/include/buf0flu.ic
M /branches/innodb+multibp/mtr/mtr0mtr.c
branches/innodb+multibp: Fix the debug assertions for flush order mutex. These
were missed in r6934.
------------------------------------------------------------------------
r6936 | sbains | 2010-04-01 02:46:52 +0300 (Thu, 01 Apr 2010) | 4 lines
Changed paths:
M /branches/innodb+multibp/sync/sync0sync.c
branches/innodb+multibp: Because now we have multiple instances of a mutex at
the same level and these mutexes can be acquired simultaneously we can't
simply check for <= level. We need to check for <= level - 1.
------------------------------------------------------------------------
r6937 | sbains | 2010-04-01 04:40:17 +0300 (Thu, 01 Apr 2010) | 5 lines
Changed paths:
M /branches/innodb+multibp/trx/trx0purge.c
branches/innodb+multibp: We need to check if the history list len is > than
some threshold not that it is evenly divisible by the some batch size. While
running tests on dscczz01 I've observed that the purge thread can't keep up
with the generation of the UNDO log records because of the faster code.
------------------------------------------------------------------------
r6938 | irana | 2010-04-01 10:15:00 +0300 (Thu, 01 Apr 2010) | 7 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0buf.c
M /branches/innodb+multibp/buf/buf0flu.c
M /branches/innodb+multibp/include/buf0buf.h
M /branches/innodb+multibp/include/buf0buf.ic
M /branches/innodb+multibp/include/buf0flu.h
M /branches/innodb+multibp/include/buf0flu.ic
M /branches/innodb+multibp/include/log0log.h
M /branches/innodb+multibp/include/sync0sync.h
M /branches/innodb+multibp/log/log0log.c
M /branches/innodb+multibp/log/log0recv.c
M /branches/innodb+multibp/mtr/mtr0mtr.c
M /branches/innodb+multibp/sync/sync0sync.c
branches/innodb+multibp
The buf_flush_order patch that was ported in from 1.1 won't work with
multiple buffer pools. This patch moves the mutex protecting order of
insertion in the flush list(s) to log_sys struct so that we can have
one global mutex protecting insertions into all flush list(s)
------------------------------------------------------------------------
r6941 | sbains | 2010-04-02 00:51:28 +0300 (Fri, 02 Apr 2010) | 4 lines
Changed paths:
M /branches/innodb+multibp/lock/lock0lock.c
branches/innodb+multibp: We should get the record heap no to check recursively
only if we are checking a record lock. Prior to this fix we were doing it for
table locks as well, this is a bug.
------------------------------------------------------------------------
r6942 | csun | 2010-04-02 02:39:10 +0300 (Fri, 02 Apr 2010) | 4 lines
Changed paths:
M /branches/innodb+multibp/ha/ha0ha.c
branches/innodb+multibp: fix compiler errors on Windows.
Move ut_ad() to after declarations for C file.
------------------------------------------------------------------------
r6943 | sbains | 2010-04-03 05:14:25 +0300 (Sat, 03 Apr 2010) | 2 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0buf.c
branches/innodb+multibp: Remove the code that created the fake buffer pool.
------------------------------------------------------------------------
r6945 | irana | 2010-04-05 23:35:29 +0300 (Mon, 05 Apr 2010) | 5 lines
Changed paths:
M /branches/innodb+multibp/lock/lock0lock.c
branches/innodb+multibp
Revert r6941 as it does not resolve the issue and we have to take
back the whole fix for bug#49047
------------------------------------------------------------------------
r6946 | irana | 2010-04-05 23:50:42 +0300 (Mon, 05 Apr 2010) | 6 lines
Changed paths:
M /branches/innodb+multibp/include/ut0ut.h
M /branches/innodb+multibp/include/ut0ut.ic
M /branches/innodb+multibp/lock/lock0lock.c
branches/innodb+multibp
Merged revisions 6932:6944 from branches/innodb+
This solely includes the reversal of fix for bug#49047
------------------------------------------------------------------------
r6947 | sbains | 2010-04-06 01:33:46 +0300 (Tue, 06 Apr 2010) | 3 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0lru.c
branches/innodb+multibp: Remove the log sys mutex acquisition when doing
buffer pool stat aggregation. A dirty read here should suffice.
------------------------------------------------------------------------
r6951 | irana | 2010-04-06 17:25:29 +0300 (Tue, 06 Apr 2010) | 5 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0buf.c
branches/innodb+mbp
Initialize the buf_page_t::buf_pool pointer when the descriptor is
allocated using buf_buddy_alloc().
------------------------------------------------------------------------
r6954 | jyang | 2010-04-06 21:24:46 +0300 (Tue, 06 Apr 2010) | 4 lines
Changed paths:
M /branches/innodb+multibp/handler/ha_innodb.cc
branches/innodb+multibp: Fix a possible null pointer of index_mapping
in a race condition.
------------------------------------------------------------------------
r6958 | sbains | 2010-04-07 00:27:44 +0300 (Wed, 07 Apr 2010) | 3 lines
Changed paths:
M /branches/innodb+multibp/include/ut0mem.h
M /branches/innodb+multibp/ut/ut0mem.c
branches/innodb+multibp: Fix part of Bug#52546. We allow ut_free() to accept
a NULL pointer and treat it as a nop.
------------------------------------------------------------------------
r6961 | jyang | 2010-04-07 10:50:03 +0300 (Wed, 07 Apr 2010) | 9 lines
Changed paths:
M /branches/innodb+multibp/handler/ha_innodb.cc
branches/innodb+multibp: Fix for bug #52580: Crash in
ha_innobase::open on executing INSERT with concurrent ALTER TABLE.
Change in MySQL bug #51557 releases the mutex LOCK_open before
ha_innobase::open(), causing racing condition for index translation
table creation. Fix it by adding dict_sys mutex for the operation.
rb://283, approved by Marko.
------------------------------------------------------------------------
r6963 | irana | 2010-04-07 19:14:10 +0300 (Wed, 07 Apr 2010) | 15 lines
Changed paths:
M /branches/innodb+multibp/handler/ha_innodb.cc
branches/innodb+multibp
Force setting of buf_pool->LRU_old_ratio by calling
buf_LRU_old_ratio_update() with adjust set to TRUE. This will make sure
that we grab the buf_pool mutex and actually adjust the
buf_pool->LRU_old pointer instead of just updating the
buf_pool->LRU_old_ratio.
Note that after this change there is no call to
buf_LRU_old_ratio_update() with adjust set to FALSE and therefore
this parameter should be removed. I am keeping it for now to first
make sure that the fix does work.
Approved by: No one. Sunny agreed with my hypothesis of the problem.
------------------------------------------------------------------------
r6964 | irana | 2010-04-07 19:59:59 +0300 (Wed, 07 Apr 2010) | 5 lines
Changed paths:
M /branches/innodb+multibp/handler/ha_innodb.cc
branches/innodb+multibp
Remove a too strong assertion on behalf of Jimmy.
------------------------------------------------------------------------
r6971 | sbains | 2010-04-09 13:23:33 +0300 (Fri, 09 Apr 2010) | 6 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0buf.c
branches/innodb+multibp: When getting the oldest (minimum) LSN value from all
the flush lists we need to acquire the flush list mutex. We were incorrectly
acquiring the buffer pool mutex.
This patch should fix a slew of bugs reported by Michael.
------------------------------------------------------------------------
r6972 | sbains | 2010-04-10 00:25:09 +0300 (Sat, 10 Apr 2010) | 5 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0buf.c
branches/innodb+multibp: We should not reset the lsn to 0 when we encounter
an empty flush list. Oldest LSN should be 0 only when all flush lists are empty.
e.g., without this fix if even one flush list was empty we would end up
breaking WAL.
------------------------------------------------------------------------
r6987 | sbains | 2010-04-14 00:14:13 +0300 (Wed, 14 Apr 2010) | 12 lines
Changed paths:
M /branches/innodb+multibp/buf/buf0buf.c
branches/innodb+multibp: When calculating the oldest_lsn we can have a
situation where we've iterated to say buffer pool 3 and another thread
adds two new dirty pages, the first to buffer pool 1 and the second to
buffer pool 4. Up to say buffer pool 3 the oldest_lsn was 0. Now, we will
end up returning the lsn at buffer pool 4 as the oldest LSN. We prevent this
by acquiring the flush order mutex.
One other future option is to calculate the min_lsn when flushing pages
from the list and maintaining a running total using atomics. That way
we can get rid of this function altogether. The atomics will only really
be required when we do parallel flushing.
------------------------------------------------------------------------
r6992 | sbains | 2010-04-14 02:45:59 +0300 (Wed, 14 Apr 2010) | 2 lines
Changed paths:
M /branches/innodb+multibp/include/ut0rbt.h
M /branches/innodb+multibp/ut/ut0rbt.c
branches/innodb+multibp: Fix copyright of the rbt code.
------------------------------------------------------------------------
