The function posix_fallocate() as well as the Linux system call
fallocate() can return EINTR when the operation was interrupted
by a signal. In that case, keep retrying the operation, except
if InnoDB shutdown has been initiated.
fil_extend_space_to_desired_size(): Use a proper type cast when
computing start_offset for the posix_fallocate() call on 32-bit systems
(where sizeof(ulint) < sizeof(os_offset_t)). This could affect 32-bit
systems when extending files that are at least 4 MiB long.
This bug existed in MariaDB 10.0 before MDEV-11520. In MariaDB 10.1
it had been fixed in MDEV-11556.
a large memory buffer on Windows
fil_extend_space_to_desired_size(), os_file_set_size(): Use calloc()
for memory allocation, and handle failures. Properly check the return
status of posix_fallocate(), and pass the correct arguments to
posix_fallocate().
On Windows, instead of extending the file by at most 1 megabyte at a time,
write a zero-filled page at the end of the file.
According to the Microsoft blog post
https://blogs.msdn.microsoft.com/oldnewthing/20110922-00/?p=9573
this will physically extend the file by writing zero bytes.
(InnoDB never uses DeviceIoControl() to set the file sparse.)
I tested that the file extension works properly with a multi-file
system tablespace, both with --innodb-use-fallocate and
--skip-innodb-use-fallocate (the default):
./mtr \
--mysqld=--innodb-use-fallocate \
--mysqld=--innodb-autoextend-increment=1 \
--mysqld=--innodb-data-file-path='ibdata1:5M;ibdata2:5M:autoextend' \
--parallel=auto --force --retry=0 --suite=innodb &
ls -lsh mysql-test/var/*/mysqld.1/data/ibdata2
(several samples while running the test)
Merge Facebook commit f981a51a47519b0ba527917887f8adc6df9ae147
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6.
This just moves some structure definitions from inside a
single .cc file to a shared .h file, with a few tweaks to
allow these structures to be shared.
On its own, it should have no actual effect. This is needed later.
Analysis: This was merge error on file fil0fil.cc. fil_system mutex was taken twice because of this.
Fix: Remove unnecessary mutex_enter and fixed the issue with slow posix_fallocate usage.
Update InnoDB to 5.6.14
Apply MySQL-5.6 hack for MySQL Bug#16434374
Move Aria-only HA_RTREE_INDEX from my_base.h to maria_def.h (breaks an assert in InnoDB)
Fix InnoDB memory leak
rb://1293
approved by: Marko Makela
There is race when dropping a single table tablespace where a reader
thread can initiate a read request before the delete flag is set and
before it is finished the deleting thread can attempt to free the
fil_node.
This patch checks the status in fil_io() to make sure that the
tablespace is not being deleted. If it is being deleted then
an error is returned instead of attempting IO.
Changes in the InnoDB codebase required to compile and
integrate the MEB codebase with MySQL 5.5.
@ storage/innobase/btr/btr0btr.c
Excluded buffer pool usage from MEB build.
buf_pool_from_bpage calls are in buf0buf.ic, and
the buffer pool functions from that file are
disabled in MEB.
@ storage/innobase/buf/buf0buf.c
Disabling more buffer pool functions unused in MEB.
@ storage/innobase/dict/dict0dict.c
Disabling dict_ind_free that is unused in MEB.
@ storage/innobase/dict/dict0mem.c
The include
#include "ha_prototypes.h"
Was causing conflicts with definitions in my_global.h
Linking C executable mysqlbackup
libinnodb.a(dict0mem.c.o): In function `dict_mem_foreign_table_name_lookup_set':
dict0mem.c:(.text+0x91c): undefined reference to `innobase_get_lower_case_table_names'
libinnodb.a(dict0mem.c.o): In function `dict_mem_referenced_table_name_lookup_set':
dict0mem.c:(.text+0x9fc): undefined reference to `innobase_get_lower_case_table_names'
libinnodb.a(dict0mem.c.o): In function `dict_mem_foreign_table_name_lookup_set':
dict0mem.c:(.text+0x96e): undefined reference to `innobase_casedn_str'
libinnodb.a(dict0mem.c.o): In function `dict_mem_referenced_table_name_lookup_set':
dict0mem.c:(.text+0xa4e): undefined reference to `innobase_casedn_str'
collect2: ld returned 1 exit status
make[2]: *** [mysqlbackup] Error 1
innobase_get_lower_case_table_names
innobase_casedn_str
are functions that are part of ha_innodb.cc that is not part of the build
dict_mem_foreign_table_name_lookup_set
function is not there in the current codebase, meaning we do not use it in MEB.
@ storage/innobase/fil/fil0fil.c
The srv_fast_shutdown variable is declared in
srv0srv.c that is not compiled in the
mysqlbackup codebase.
This throws an undeclared error.
From the Manual
---------------
innodb_fast_shutdown
--------------------
The InnoDB shutdown mode. The default value is 1
as of MySQL 3.23.50, which causes a “fast� shutdown
(the normal type of shutdown). If the value is 0,
InnoDB does a full purge and an insert buffer merge
before a shutdown. These operations can take minutes,
or even hours in extreme cases. If the value is 1,
InnoDB skips these operations at shutdown.
This ideally does not matter from mysqlbackup
@ storage/innobase/ha/ha0ha.c
In file included from /home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/ha/ha0ha.c:34:0:
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/include/btr0sea.h:286:17: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
make[2]: *** [CMakeFiles/innodb.dir/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/ha/ha0ha.c.o] Error 1
make[1]: *** [CMakeFiles/innodb.dir/all] Error 2
make: *** [all] Error 2
# include "sync0rw.h" is excluded from hotbackup compilation in dict0dict.h
This causes extern rw_lock_t* btr_search_latch_temp; to throw a failure because
the definition of rw_lock_t is not found.
@ storage/innobase/include/buf0buf.h
Excluding buffer pool functions that are unused from the
MEB codebase.
@ storage/innobase/include/buf0buf.ic
replicated the exclusion of
#include "buf0flu.h"
#include "buf0lru.h"
#include "buf0rea.h"
by looking at the current codebase in <meb-trunk>/src/innodb
@ storage/innobase/include/dict0dict.h
dict_table_x_lock_indexes, dict_table_x_unlock_indexes, dict_table_is_corrupted,
dict_index_is_corrupted, buf_block_buf_fix_inc_func are unused in MEB and was
leading to compilation errors and hence excluded.
@ storage/innobase/include/dict0dict.ic
dict_table_x_lock_indexes, dict_table_x_unlock_indexes, dict_table_is_corrupted,
dict_index_is_corrupted, buf_block_buf_fix_inc_func are unused in MEB and was
leading to compilation errors and hence excluded.
@ storage/innobase/include/log0log.h
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/include/log0log.h: At top level:
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/include/log0log.h:767:2: error: expected specifier-qualifier-list before â⠂¬Ëœmutex_t’
mutex_t definitions were excluded as seen from ambient code
hence excluding definition for log_flush_order_mutex also.
@ storage/innobase/include/os0file.h
Bug in InnoDB code, create_mode should have been create.
@ storage/innobase/include/srv0srv.h
In file included from /home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/buf/buf0buf.c:50:0:
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/include/srv0srv.h: At top level:
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/include/srv0srv.h:120:16: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘srv_use_native_aio’
srv_use_native_aio - we do not use native aio of the OS anyway from MEB. MEB does not compile
InnoDB with this option. Hence disabling it.
@ storage/innobase/include/trx0sys.h
[ 56%] Building C object CMakeFiles/innodb.dir/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/trx/trx0sys.c.o
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/trx/trx0sys.c: In function ‘trx_sys_read_file_format_id’:
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/trx/trx0sys.c:1499:20: error: ‘TRX_SYS_FILE_FORMAT_TAG_MAGIC_N’ undeclared (first use in this function)
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/trx/trx0sys.c:1499:20: note: each undeclared identifier is reported only once for each function it appears in
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/trx/trx0sys.c: At top level:
/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/include/buf0buf.h:607:1: warning: ‘buf_block_buf_fix_inc_func’ declared ‘static’ but never defined
make[2]: *** [CMakeFiles/innodb.dir/home/narayanan/mysql-server/mysql-5.5-meb-rel3.8-innodb-integration-1/storage/innobase/trx/trx0sys.c.o] Error 1
unused calls excluded to enable compilation
@ storage/innobase/mem/mem0dbg.c
excluding #include "ha_prototypes.h" that lead to definitions in ha_innodb.cc
@ storage/innobase/os/os0file.c
InnoDB not compiled with aio support from MEB anyway. Hence excluding this from
the compilation.
@ storage/innobase/page/page0zip.c
page0zip.c:(.text+0x4e9e): undefined reference to `buf_pool_from_block'
collect2: ld returned 1 exit status
buf_pool_from_block defined in buf0buf.ic, most of the file is excluded for compilation of MEB
@ storage/innobase/ut/ut0dbg.c
excluding #include "ha_prototypes.h" since it leads to definitions in ha_innodb.cc
innobase_basename(file) is defined in ha_innodb.cc. Hence excluding that also.
@ storage/innobase/ut/ut0ut.c
cal_tm unused from MEB, was leading to earnings, hence disabling for MEB.
rb://942
approved by: Marko Makela
We don't need to scan LRU for dropping AHI entries when DROPing a table.
AHI entries are already removed when we free up extents for the btree.
rb://914
approved by: Marko Makela
Poll in fil_rename_tablespace() after setting ::stop_ios flag can
result in a hang because the other thread actually dispatching the IO
won't wake IO helper threads or flush the tablespace before starting
wait in fil_mutex_enter_and_prepare_for_io().
This bug ensures that a live downgrade from an InnoDB 5.6 with WL5756 and
a database created with innodb-page-size=8k or 4k will make a version 5.5
installation politely refuse to start. Instead of crashing or giving some
indication about a corrupted database, it will indicate the page size
difference.
This patch takes only that part of the Wl5756 patch that is needed to
protect against opening a tablespace that is stamped with a different
page size.
It also contains the change in dict_index_find_on_id_low() just in case
a database with another page size is created by recompiling a pre-WL5756
InnoDB.
node->modification_counter == node->flush_counter
This bug (an assertion failure during a crash-like shutdown)
was introduced by the fix of
Bug#12323643 Clean up the InnoDB thread shutdown and assertions (WL#5136).
fil_node_close_file(): Relax the failing assertion
when innodb_fast_shutdown=2.
On shutdown, do not exit threads in os_event_wait(). This method of
exiting was only used by the I/O handler threads. Exit them on a
higher level.
os_event_wait_low(), os_event_wait_time_low(): Do not exit on shutdown.
os_thread_exit(), ut_dbg_assertion_failed(), ut_print_timestamp(): Add
attribute cold, so that GCC knows that these functions are rarely
invoked and can be optimized for size.
os_aio_linux_collect(): Return on shutdown.
os_aio_linux_handle(), os_aio_simulated_handle(), os_aio_windows_handle():
Set *message1 = *message2 = NULL and return TRUE on shutdown.
fil_aio_wait(): Return on shutdown.
logs_empty_and_mark_files_at_shutdown(): Even in very fast shutdown
(innodb_fast_shutdown=2), allow the background threads to exit, but
skip the flushing and log checkpointing.
innobase_shutdown_for_mysql(): Always wait for all the threads to exit.
rb:633 approved by Sunny Bains
ibuf_inside(), ibuf_enter(), ibuf_exit(): Add the parameter mtr. The
flag is no longer kept in the thread-local storage but in the
mini-transaction (mtr->inside_ibuf).
mtr_start(): Clean up the comment and remove the unused return value.
mtr_commit(): Assert !ibuf_inside(mtr) in debug builds.
ibuf_mtr_start(): Like mtr_start(), but sets the flag.
ibuf_mtr_commit(), ibuf_btr_pcur_commit_specify_mtr(): Wrappers that
assert ibuf_inside().
buf_page_get_zip(), buf_page_init_for_read(),
buf_read_ibuf_merge_pages(), fil_io(), ibuf_free_excess_pages(),
ibuf_contract_ext(): Remove assertions on ibuf_inside(), because a
mini-transaction is not available.
buf_read_ahead_linear(): Add the parameter inside_ibuf.
ibuf_restore_pos(): When this function returns FALSE, it commits mtr
and must therefore do ibuf_exit(mtr).
ibuf_delete_rec(): This function commits mtr and must therefore do
ibuf_exit(mtr).
ibuf_rec_get_page_no(), ibuf_rec_get_space(), ibuf_rec_get_info(),
ibuf_rec_get_op_type(), ibuf_build_entry_from_ibuf_rec(),
ibuf_rec_get_volume(), ibuf_get_merge_page_nos(),
ibuf_get_volume_buffered_count(), ibuf_get_entry_counter_low(): Add
the parameter mtr in debug builds, for asserting ibuf_inside(mtr).
rb:585 approved by Sunny Bains
Do not disable InnoDB inlining when UNIV_DEBUG is defined. The
inlining is now solely controlled by the preprocessor symbol
UNIV_MUST_NOT_INLINE and by any compiler options.
mtr_memo_contains(): Add an explicit type conversion from void*, so
that the function can be compiled by a C++ compiler. Previously, this
function was never seen by the C++ compiler, because it is only
present in UNIV_DEBUG builds and InnoDB inlining used to be disabled.
buf_flush_validate_skip(): A wrapper that skips most calls of
buf_flush_validate_low(). Invoked by debug assertions in
buf_flush_insert_into_flush_list() and buf_flush_remove().
fil_validate_skip(): A wrapper that skips most calls of
fil_validate(). Invoked by debug assertions in fil_io() and fil_io_wait().
os_aio_validate_skip(): A wrapper that skips most calls of
os_aio_validate(). Invoked by debug assertions in
os_aio_func(), os_aio_windows_handle() and os_aio_simulated_handle.
os_get_os_version(): Only include this function if __WIN__ is defined.
sync_array_deadlock_step(): Slight optimizations. This function is a
major CPU consumer in UNIV_SYNC_DEBUG builds.
This patch doesn't get rid of the need to acquire the dict_sys->mutex but
reduces the need to keep the mutex locked for the duration of the query
to fsp_get_available_space_in_free_extents() from ha_innobase::info().
rb://390.
------------------------------------------------------------
revno: 3515
revision-id: marko.makela@oracle.com-20100621094008-o9fa153s3f09merw
parent: vasil.dimov@oracle.com-20100618085155-xnm469cbhedqea96
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: 5.1-innodb
timestamp: Mon 2010-06-21 12:40:08 +0300
message:
Bug #54658: InnoDB: Warning: allocated tablespace %lu, old maximum was 0
dict_check_tablespaces_and_store_max_id(): Initialize max_space_id
and fil_system->max_assigned_id from DICT_HDR_MAX_SPACE_ID.
fil_space_create(): Suppress the warning unless !recv_recovery_on
(do not complain while applying the redo log).
------------------------------------------------------------
revno: 3479
revision-id: marko.makela@oracle.com-20100524110439-fazi70rlmt07tzd9
parent: vasil.dimov@oracle.com-20100520133157-42uk5q3pp0vsinac
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: 5.1-innodb
timestamp: Mon 2010-05-24 14:04:39 +0300
message:
Bug#53578: assert on invalid page access, in fil_io()
Store the max_space_id in the data dictionary header in order to avoid
space_id reuse.
DICT_HDR_MIX_ID: Renamed to DICT_HDR_MAX_SPACE_ID, DICT_HDR_MIX_ID_LOW.
dict_hdr_get_new_id(): Return table_id, index_id, space_id or a subset of them.
fil_system_t: Add ibool space_id_reuse_warned.
fil_create_new_single_table_tablespace(): Get the space_id from the caller.
fil_space_create(): Issue a warning if the fil_system->max_assigned_id
is exceeded.
fil_assign_new_space_id(): Return TRUE/FALSE and take a pointer to the
space_id as a parameter. Make the function public.
fil_init(): Initialize all fil_system fields by mem_zalloc(). Remove
explicit initializations of certain fields to 0 or NULL.
not address the printouts issue
Detailed revision comments:
r6310 | marko | 2009-12-15 15:23:54 +0200 (Tue, 15 Dec 2009) | 30 lines
branches/5.1: Merge r4922 from branches/zip.
This the fix for the first part of Bug #41609 from InnoDB Plugin to
the built-in InnoDB in MySQL 5.1. This allows InnoDB Hot Backup to
back up a database while the built-in InnoDB in MySQL 5.1 is creating
temporary tables. (This fix does not address the printouts about
missing .ibd files for temporary tables at InnoDB startup, which was
committed to branches/zip in r6252.)
rb://219 approved by Sunny Bains.
branches/zip: Distinguish temporary tables in MLOG_FILE_CREATE.
This addresses Mantis Issue #23 in InnoDB Hot Backup and some
of MySQL Bug #41609.
In MLOG_FILE_CREATE, we need to distinguish temporary tables, so that
InnoDB Hot Backup can work correctly. It turns out that we can do this
easily, by using a bit of the previously unused parameter for page number.
(The page number parameter of MLOG_FILE_CREATE has been written as 0
ever since MySQL 4.1, which introduced MLOG_FILE_CREATE.)
MLOG_FILE_FLAG_TEMP: A flag for indicating a temporary table in
the page number parameter of MLOG_FILE_ operations.
fil_op_write_log(): Add the parameter log_flags.
fil_op_log_parse_or_replay(): Add the parameter log_flags.
Do not replay MLOG_FILE_CREATE when MLOG_FILE_FLAG_TEMP is set in log_flags.
This only affects ibbackup --apply-log. InnoDB itself never replays file
operations.
