rb://816
approved by: Marko Makela
The title is misleading. This bug was actually introduced by
bug 12635227 and was unearthed by a later optimization.
We need to free buf_page_t structs that we are allocating using
malloc() at shutdown.
a.k.a. Bug#7975 deadlock without any locking, simple select and update
Bug#7975 was reintroduced when the storage engine API was made
pluggable in MySQL 5.1. Instead of looking at thd->lex directly, we
rely on handler::extra(). But, we were looking at the wrong extra()
flag, and we were ignoring the TRX_DUP_REPLACE flag in places where we
should obey it.
innodb_replace.test: Add tests for hopefully all affected statement
types, so that bug should never ever resurface. This kind of tests
should have been added when fixing Bug#7975 in MySQL 5.0.3 in the
first place.
rb:806 approved by Sunny Bains
btr_pcur_restore_position_func(): When the cursor was positioned at
the tree infimum or supremum, initialize pos_state and latch_mode. The
assertion failed, because pos_state was BTR_PCUR_WAS_POSITIONED. In
the test failure of WL#5874, the purge thread attempted to restore the
cursor position on the infimum record (the clustered index was empty).
btr_pcur_detach(), btr_pcur_is_detached(): Unused functions, remove.
rb:804 approved by Inaam Rana
row_rename_table_for_mysql(): Return DB_ERROR instead of DB_SUCCESS
when fil_rename_tablespace() returns an error. This bug was introduced
in the InnoDB Plugin.
Approved by Sunny Bains over IM.
Bug#12612184 RACE CONDITION AFTER BTR_CUR_PESSIMISTIC_UPDATE()
The fix introduced potentially more severe crash recovery problems
than the bug causes. Revert the fix for now.
This was an attempt to address problems with the Bug#12612184 fix.
Even with this follow-up fix, crash recovery can be broken.
Let us fix the bug later.
In the ON UPDATE CASCADE clause of FOREIGN KEY constraints, the
calculated update vector was not fully initialized. This bug was
introduced in the InnoDB Plugin when implementing support for
ROW_FORMAT=DYNAMIC.
Additionally, the data type information was not initialized, but
apparently it has never been needed in this case. Nevertheless, it is
not good programming practice to pass uninitialized values around.
calc_row_difference(): Declare the update field uninitialized in
Valgrind. Copy the data type information as well, except when the
field is SQL NULL. In the built-in InnoDB, initialize
ufield->extern_storage = FALSE (an initialization bug that had gone
unnoticed this far). The InnoDB Plugin and later have this flag to
dfield_t and have always initialized it properly.
row_ins_cascade_calc_update_vec(): Reduce the scope of some
pointers. Initialize orig_len. (This caused the bug in InnoDB Plugin
and later.)
row_ins_foreign_check_on_constraint(): Simplify a condition. Declare
the update vector uninitialized.
rb:771 approved by Jimmy Yang
btr_record_not_null_field_in_rec(): Remove the parameter rec.
Use rec_offs_nth_sql_null() instead of rec_get_nth_field().
rb:788 approved by Jimmy Yang
USING MYISAM_USE_MMAP ON WINDOWS
When OPTIMIZE/REPAIR TABLE is switching to new data file,
old data file is removed while memory mapping is still
active.
With 5.1 implementation of nt_share_delete() it is not
permitted to remove mmaped file.
This fix disables memory mapping for mi_repair() operations.
mysql-test/r/myisam.result:
A test case for BUG#11757032.
mysql-test/t/myisam.test:
A test case for BUG#11757032.
storage/myisam/ha_myisam.cc:
mi_repair*() functions family use file I/O even if memory
mapping is available.
Since mixing mmap I/O and file I/O may cause various artifacts,
memory mapping must be disabled.
storage/myisam/mi_delete_all.c:
Clean-up: do not attempt to remap file after truncate, since
there is nothing to map.
hash index at shutdown
btr_search_disable(): Just drop the entire adaptive hash index,
without dropping every record separately.
buf_pool_clear_hash_index(): Renamed and simplified from
buf_pool_drop_hash_index(). Set block->index = NULL for every block in
the buffer pool. Do not release the btr_search_latch. The caller will
have to adjust other data structures.
Remove block->is_hashed. It is redundant, should be always equal to
block->index != NULL.
Remove btr_search_fully_disabled, btr_search_enabled_mutex, and
SYNC_SEARCH_SYS_CONF. We drop the AHI in one pass, without releasing
the btr_search_latch in between.
Replace void* with const rec_t* and add assertions on btr_search_latch
and btr_search_enabled to ha0ha.h, ha0ha.ic, ha0ha.c.
page_set_max_trx_id(): Ignore the adaptive hash index. I forgot to
push this in rb:750.
btr0sea.c: Always after acquiring btr_search_latch, check for
block->index==NULL or !btr_search_enabled. We can now set
block->index=NULL while only holding btr_search_latch in exclusive
mode. Always acquire btr_search_latch before reading block->index,
except in shortcuts when testing for block->index == NULL.
ha_clear(), ha_search(): Unused function, remove.
buf_page_peek_if_search_hashed(): Remove. This function may avoid
latching a page at the cost of doing a duplicate buf_pool->page_hash
lookup.
rb:775 approved by Inaam Rana
rw_lock_x_lock_func(): Assert that the thread is not already holding
the lock in a conflicting mode (RW_LOCK_SHARED).
rw_lock_s_lock_func(): Assert that the thread is not already holding
the lock in a conflicting mode (RW_LOCK_EX).
Bug 12980094 - ASSERTION IN INNODB DETECTED IN RQG_PARTITION_DDL
Bug 13034534 - RQG TESTS FAIL ON WINDOWS WITH CRASH NEAR RW_LOCK_DEBUG_PRINT
All access to struct rw_lock_debug_struct must be protected by rw_lock_debug_mutex_enter().
CMakeLists.txt:
1. add -DSAFEMALLOC -DSAFE_MUTEX in the top-level CMakeLists.txt
don't force plugins to copy-paste these lines in their CMakeLists.txt
2.1 search plugin/* for plugins (not only storage/*),
2.2 recognize MYSQL_PLUGIN (not only MYSQL_STORAGE_ENGINE),
2.3 extract library names from the plug.in (don't force library names to
be ha_<engine>.dll and <engine>.lib)
include/mysql/plugin.h:
define MYSQL_PLUGIN_EXPORT appropriately
(backport from 5.5)
libmysqld/CMakeLists.txt:
remove unnecessary workaround
plugin/fulltext/CMakeLists.txt:
build fulltext example plugin on windows
storage/maria/CMakeLists.txt:
The library is called libmaria_s.lib, not maria.lib
storage/maria/unittest/CMakeLists.txt:
The library is called libmaria_s.lib, not maria.lib
storage/myisam/CMakeLists.txt:
The library is called libmyisam_s.lib, not myisam.lib
storage/mysql_storage_engine.cmake:
introduce MYSQL_PLUGIN macro.
don't force library names to be ha_<engine>.dll and <engine>.lib
storage/xtradb/CMakeLists.txt:
remove a condition from include
win/README:
don't use deprecated syntax
win/configure-mariadb.sh:
don't use deprecated syntax
win/configure.js:
1. support MYSQL_PLUGIN in addition to MYSQL_STORAGE_ENGINE.
2. support plugin/* in addition to storage/*
CREATE_TIME IN INFORMATION_SC
It was impossible to determine MEMORY table creation time,
since it wasn't stored/exposed.
With this patch creation time is saved and it is available via
I_S.TABLES.CREATE_TIME.
Note: it was decided that additional analysis is required before
implementing UPDATE_TIME. Thus this patch doesn't store UPDATE_TIME.
Replace part of the patch that Kevin apparently forgot to push.
Fix the bug also in the built-in InnoDB of MySQL 5.1.
I cannot explain why the test case was not failing without the
full patch.
This was rb:762, approved by me.
The problem occurred when indexes are added between the time that an
UNDO record is created and the time that the purge thread comes around
and deletes the old secondary index entries. The purge thread would
hit an assert when trying to build a secondary index entry for
searching. The problem was that the old value of those fields were not
in the UNDO record since they were not part of an index when the UPDATE
occured.
A test case was added to innodb-index.test.
FULLTEXT INDEXES
myisamchk may create incorrect fulltext index for compressed
tables. Incorrect data pointer size was used while creating
fulltext index.
mysql-test/r/myisampack.result:
A test case for BUG#11761180.
mysql-test/t/myisampack.test:
A test case for BUG#11761180.
storage/myisam/ft_boolean_search.c:
rec_reflength on share may have adjustments required for
compressed tables and must be used instead of rec_reflength
on base info.
storage/myisam/ft_nlq_search.c:
rec_reflength on share may have adjustments required for
compressed tables and must be used instead of rec_reflength
on base info.
storage/myisam/mi_check.c:
rec_reflength on share may have adjustments required for
compressed tables and must be used instead of rec_reflength
on base info.
storage/myisam/mi_write.c:
rec_reflength on share may have adjustments required for
compressed tables and must be used instead of rec_reflength
on base info.
when buffered changes are to be discarded
sync_thread_add_level(level = SYNC_IBUF_INDEX_TREE): Relax a too strict
condition that the thread must not be holding locks below SYNC_FSP_PAGE.
It is perfectly valid to hold any latch above SYNC_IBUF_INDEX_TREE when
acquiring the insert buffer tree latch.
when buffered changes are to be discarded
ibuf_add_free_page(): Lower the latching order of the newly allocated page
to SYNC_IBUF_TREE_NODE_NEW after latching the insert buffer tree root.
This bug always was bogus UNIV_SYNC_DEBUG alarm. The function
buf_block_dbg_add_level() is a no-op unless UNIV_SYNC_DEBUG is defined.
When there is a secondary index on a column prefix of an externally
stored column and an entry in the secondary index is shorter than the
reserved prefix length, it should mean that the secondary index entry
is holding the complete column value. When comparing this secondary
index column value to the column in the clustered index row, we must
compare the entire prefix that was fetched from the clustered
index. The bug was that we would just compare that the column in the
clustered index starts with the value found in the secondary index
column.
This bug affects only the InnoDB Barracuda formats (ROW_FORMAT=DYNAMIC
and ROW_FORMAT=COMPRESSED), in which columns that are stored off-page
in the clustered index do not contain any prefix in the clustered
index record.
row_sel_sec_rec_is_for_blob(): Add the parameter prefix_len, for
ifield->prefix_len. Add some assertions.
Sorry, I did not manage to produce a test case. This patch does
produce correct results on the data set that Michael isolated on our
test machine. That was with the purge and background rollback
suspended, because they would make the bug go away.
rb:760 approved by Sunny Bains
This change is a followup to
vasil.dimov@oracle.com-20110907145810-v98kldmho23vhhic
which triggered the usage of the prefetch and valgrind tests spat lots of
warnings.
The prefetch code will be removed.
Discussed with: Marko (over IM)
InnoDB acquires an x-latch on btr_search_latch for certain in-place updates
that do affect the adaptive hash index. These operations do not really need
to be protected by the btr_search_latch:
* updating DB_TRX_ID
* updating DB_ROLL_PTR
* updating PAGE_MAX_TRX_ID
* updating the delete-mark flag
rb:750 approved by Sunny Bains
Tweak the faulty UNIV_SYNC_DEBUG diagnostics a little bit more.
ibuf_add_free_page(): Lower the latching order of the newly allocated page
only after acquiring the ibuf_mutex.
DROP_TABLE_PROC().
With this change I observe a speedup from 6.2s to 0.1s when executing
DROP_TABLE_PROC() during DROP TABLE with 512 foreign keys, like what
is being done in innodb_bug56143.test
This fixes "Bug#11765460 DROP TABLE USES INEFFICIENT METHODS TO REMOVE
FKS/INDEXES FROM INNODB SYS TABLES"
Reviewed by: Marko
buf_buddy_relocate(): The ut_time_us() function is needed for
statistics, calculating the total time spent on relocating blocks.
Until now, we invoked ut_time_us() every time buf_buddy_relocate() was
called. Fix: Only call ut_time_us() when the block can be relocated.
After this fix, the reported relocated_usec will no longer include the
time for the page_hash lookup and for acquiring the block mutex.
Approved by Sunny Bains on IM
The original fix was accidentally pushed to mysql-5.1 after the 5.1.59
clone-off in bzr revision id
marko.makela@oracle.com-20110829081642-z0w992a0mrc62s6w with thne fix
of Bug#12704861 Corruption after a crash during BLOB update.
It was pushed separately to mysql-5.5 in bzr revision id
marko.makela@oracle.com-20110901184804-2901f6qmuro3jas8.
trx_undo_report_row_operation(): If the page for which the undo log
was too big was empty, commit and start the mini-transaction before
acquiring the rollback segment mutex and freeing the undo page. This
is necessary, because the mini-transaction may be holding lower-order
latches in the levels SYNC_FSP and SYNC_FSP_PAGE.
trx_undo_erase_page_end(): Erase also empty pages, because
trx_undo_report_row_operation() needs to commit the mini-transaction
before freeing the empty page.
rb:756 approved by Sunny Bains
The fix of Bug#12612184 broke crash recovery. When a record that
contains off-page columns (BLOBs) is updated, we must first write redo
log about the BLOB page writes, and only after that write the redo log
about the B-tree changes. The buggy fix would log the B-tree changes
first, meaning that after recovery, we could end up having a record
that contains a null BLOB pointer.
Because we will be redo logging the writes off the off-page columns
before the B-tree changes, we must make sure that the pages chosen for
the off-page columns are free both before and after the B-tree
changes. In this way, the worst thing that can happen in crash
recovery is that the BLOBs are written to free pages, but the B-tree
changes are not applied. The BLOB pages would correctly remain free in
this case. To achieve this, we must allocate the BLOB pages in the
mini-transaction of the B-tree operation. A further quirk is that BLOB
pages are allocated from the same file segment as leaf pages. Because
of this, we must temporarily "hide" any leaf pages that were freed
during the B-tree operation by "fake allocating" them prior to writing
the BLOBs, and freeing them again before the mtr_commit() of the
B-tree operation, in btr_mark_freed_leaves().
btr_cur_mtr_commit_and_start(): Remove this faulty function that was
introduced in the Bug#12612184 fix. The problem that this function was
trying to address was that when we did mtr_commit() the BLOB writes
before the mtr_commit() of the update, the new BLOB pages could have
overwritten clustered index B-tree leaf pages that were freed during
the update. If recovery applied the redo log of the BLOB writes but
did not see the log of the record update, the index tree would be
corrupted. The correct solution is to make the freed clustered index
pages unavailable to the BLOB allocation. This function is also a
likely culprit of InnoDB hangs that were observed when testing the
Bug#12612184 fix.
btr_mark_freed_leaves(): Mark all freed clustered index leaf pages of
a mini-transaction allocated (nonfree=TRUE) before storing the BLOBs,
or freed (nonfree=FALSE) before committing the mini-transaction.
btr_freed_leaves_validate(): A debug function for checking that all
clustered index leaf pages that have been marked free in the
mini-transaction are consistent (have not been zeroed out).
btr_page_alloc_low(): Refactored from btr_page_alloc(). Return the
number of the allocated page, or FIL_NULL if out of space. Add the
parameter "mtr_t* init_mtr" for specifying the mini-transaction where
the page should be initialized, or if this is a "fake allocation"
(init_mtr=NULL) by btr_mark_freed_leaves(nonfree=TRUE).
btr_page_alloc(): Add the parameter init_mtr, allowing the page to be
initialized and X-latched in a different mini-transaction than the one
that is used for the allocation. Invoke btr_page_alloc_low(). If a
clustered index leaf page was previously freed in mtr, remove it from
the memo of previously freed pages.
btr_page_free(): Assert that the page is a B-tree page and it has been
X-latched by the mini-transaction. If the freed page was a leaf page
of a clustered index, link it by a MTR_MEMO_FREE_CLUST_LEAF marker to
the mini-transaction.
btr_store_big_rec_extern_fields_func(): Add the parameter alloc_mtr,
which is NULL (old behaviour in inserts) and the same as local_mtr in
updates. If alloc_mtr!=NULL, the BLOB pages will be allocated from it
instead of the mini-transaction that is used for writing the BLOBs.
fsp_alloc_from_free_frag(): Refactored from
fsp_alloc_free_page(). Allocate the specified page from a partially
free extent.
fseg_alloc_free_page_low(), fseg_alloc_free_page_general(): Add the
parameter "mtr_t* init_mtr" for specifying the mini-transaction where
the page should be initialized, or NULL if this is a "fake allocation"
that prevents the reuse of a previously freed B-tree page for BLOB
storage. If init_mtr==NULL, try harder to reallocate the specified page
and assert that it succeeded.
fsp_alloc_free_page(): Add the parameter "mtr_t* init_mtr" for
specifying the mini-transaction where the page should be initialized.
Do not allow init_mtr == NULL, because this function is never to be
used for "fake allocations".
mtr_t: Add the operation MTR_MEMO_FREE_CLUST_LEAF and the flag
mtr->freed_clust_leaf for quickly determining if any
MTR_MEMO_FREE_CLUST_LEAF operations have been posted.
row_ins_index_entry_low(): When columns are being made off-page in
insert-by-update, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass
the mini-transaction as the alloc_mtr to
btr_store_big_rec_extern_fields(). Finally, invoke
btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages.
row_build(): Correct a comment, and add a debug assertion that a
record that contains NULL BLOB pointers must be a fresh insert.
row_upd_clust_rec(): When columns are being moved off-page, invoke
btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as
the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke
btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages.
buf_reset_check_index_page_at_flush(): Remove. The function
fsp_init_file_page_low() already sets
bpage->check_index_page_at_flush=FALSE.
There is a known issue in tablespace extension. If the request to
allocate a BLOB page leads to the tablespace being extended, crash
recovery could see BLOB writes to pages that are off the tablespace
file bounds. This should trigger an assertion failure in fil_io() at
crash recovery. The safe thing would be to write redo log about the
tablespace extension to the mini-transaction of the BLOB write, not to
the mini-transaction of the record update. However, there is no redo
log record for file extension in the current redo log format.
rb:693 approved by Sunny Bains
The bug was that when using bulk insert combined with lock table, we intitalized the io cache with the wrong file position.
This fixed a bug where MariaDB could not read in a table dump done with mysqldump.
mysql-test/suite/maria/r/locking.result:
Test case for locking + write cache bug
mysql-test/suite/maria/t/locking.test:
Test case for locking + write cache bug
storage/maria/ma_extra.c:
Initialize write cache used with bulk insert to correct file length.
(The old code didn't work if one was using LOCK TABLE for the given table).
discarded in buf_page_create()
This bug turned out to be a false alarm, a bug in the UNIV_SYNC_DEBUG
diagnostic code. Because of this, the patch was not backported to the
built-in InnoDB in MySQL 5.1. Furthermore, there is no test case for
InnoDB Plugin in MySQL 5.1, because the delete buffering in MySQL 5.5
makes triggering the failure much easier.
When a freed page for which there exist orphaned buffered changes is
allocated and reused for something else, buf_page_create() will discard
the buffered changes by invoking ibuf_merge_or_delete_for_page().
This would violate the InnoDB latching order.
Tweak the latching order as follows. Move SYNC_IBUF_MUTEX below
SYNC_FSP_PAGE, where it logically belongs, and assign new latching
levels for the ibuf->index->lock and the insert buffer B-tree pages:
#define SYNC_IBUF_MUTEX 370 /* ibuf_mutex */
#define SYNC_IBUF_INDEX_TREE 360
#define SYNC_IBUF_TREE_NODE_NEW 359
#define SYNC_IBUF_TREE_NODE 358
btr_block_get(), btr_page_get(): In UNIV_SYNC_DEBUG, add the parameter
"index" for determining the appropriate latching order
(SYNC_IBUF_TREE_NODE or SYNC_TREE_NODE).
btr_page_alloc_for_ibuf(), btr_create(): Use SYNC_IBUF_TREE_NODE_NEW
instead of SYNC_TREE_NODE_NEW for insert buffer pages.
btr_cur_search_to_nth_level(), btr_pcur_restore_position_func(): Use
SYNC_IBUF_TREE_NODE instead of SYNC_TREE_NODE for insert buffer pages.
btr_search_guess_on_hash(): Assert that the index is not an insert buffer tree.
dict_index_add_to_cache(): Use SYNC_IBUF_INDEX_TREE for the insert
buffer tree (ibuf->index->lock).
ibuf0ibuf.c: Use SYNC_IBUF_TREE_NODE or SYNC_IBUF_TREE_NODE_NEW for
all B-tree pages.
ibuf_merge_or_delete_for_page(): Assert that the user page is
BUF_IO_READ fixed. Only in this way it is OK to latch it as
SYNC_IBUF_TREE_NODE instead of the proper SYNC_TREE_NODE (which would
violate the changed latching order).
sync_thread_add_level(): Remove the special tweak for
SYNC_IBUF_MUTEX. Add rules for the added latching levels.
rb:591 approved by Jimmy Yang
The bug was a wrong check in aria_chk; The table was fine.
storage/maria/ma_bitmap.c:
Print whole bitmap to find errors in last bitmap
storage/maria/ma_check.c:
Fixed wrong test if bitmap was overallocated.
row_sel_field_store_in_mysql_format(): Do not pad the unused part of
the buffer reserved for a True VARCHAR column (introduced in 5.0.3).
Add Valgrind instrumentation ensuring that the unused part will be
flagged uninitialized.
row_sel_copy_cached_field_for_mysql(): New function: Copy a field
that is in the MySQL row format, not copying the unused tail of
VARCHAR columns.
row_sel_pop_cached_row_for_mysql(): Invoke
row_sel_copy_cached_field_for_mysql() for copying fields.
When the row is long, copy it field-by-field.
rb:715 approved by Inaam Rana
