Commit graph

61 commits

Author SHA1 Message Date
Marko Mäkelä
fa57479fcd Merge 10.0 into 10.1 2017-06-12 14:26:32 +03:00
Jan Lindström
58c56dd7f8 MDEV-12610: MariaDB start is slow
Problem appears to be that the function fsp_flags_try_adjust()
is being unconditionally invoked on every .ibd file on startup.
Based on performance investigation also the top function
fsp_header_get_crypt_offset() needs to addressed.

Ported implementation of fsp_header_get_encryption_offset()
function from 10.2 to fsp_header_get_crypt_offset().

Introduced a new function fil_crypt_read_crypt_data()
to read page 0 if it is not yet read.

fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd
file is not read on startup we need to check has page 0 read
from space that we investigate for key rotation, if it is not read
we read it.

fil_space_crypt_get_status(): Now that page 0 for every .ibd
file is not read on startup here also we need to read page 0
if it is not yet read it. This is needed
as tests use IS query to wait until background encryption
or decryption has finished and this function is used to
produce results.

fil_crypt_thread(): Add is_stopping condition for tablespace
so that we do not rotate pages if usage of tablespace should
be stopped. This was needed for failure seen on regression
testing.

fil_space_create: Remove page_0_crypt_read and extra
unnecessary info output.

fil_open_single_table_tablespace(): We call fsp_flags_try_adjust
only when when no errors has happened and server was not started
on read only mode and tablespace validation was requested or
flags contain other table options except low order bits to
FSP_FLAGS_POS_PAGE_SSIZE position.

fil_space_t::page_0_crypt_read removed.

Added test case innodb-first-page-read to test startup when
encryption is on and when encryption is off to check that not
for all tables page 0 is read on startup.
2017-06-09 13:15:39 +03:00
Marko Mäkelä
fbeb9489cd Cleanup of MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M
The doublewrite buffer pages must fit in the first InnoDB system
tablespace data file. The checks that were added in the initial patch
(commit 112b21da37)
were at too high level and did not cover all cases.

innodb.log_data_file_size: Test all innodb_page_size combinations.

fsp_header_init(): Never return an error. Move the change buffer creation
to the only caller that needs to do it.

btr_create(): Clean up the logic. Remove the error log messages.

buf_dblwr_create(): Try to return an error on non-fatal failure.
Check that the first data file is big enough for creating the
doublewrite buffers.

buf_dblwr_process(): Check if the doublewrite buffer is available.
Display the message only if it is available.

recv_recovery_from_checkpoint_start_func(): Remove a redundant message
about FIL_PAGE_FILE_FLUSH_LSN mismatch when crash recovery has already
been initiated.

fil_report_invalid_page_access(): Simplify the message.

fseg_create_general(): Do not emit messages to the error log.

innobase_init(): Revert the changes.

trx_rseg_create(): Refactor (no functional change).
2017-06-08 11:55:47 +03:00
Jan Lindström
d03abc71a4 MDEV-12609: Allow suppression of InnoDB log messages about reserving extents
Removed this output.
2017-06-08 10:34:10 +03:00
Jan Lindström
112b21da37 MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M;
Problem was that all doublewrite buffer pages must fit to first
system datafile.

Ported commit 27a34df7882b1f8ed283f22bf83e8bfc523cbfde
Author: Shaohua Wang <shaohua.wang@oracle.com>
Date:   Wed Aug 12 15:55:19 2015 +0800

    BUG#21551464 - SEGFAULT WHILE INITIALIZING DATABASE WHEN
    INNODB_DATA_FILE SIZE IS SMALL

To 10.1 (with extended error printout).

btr_create(): If ibuf header page allocation fails report error and
return FIL_NULL. Similarly if root page allocation fails return a error.

dict_build_table_def_step: If fsp_header_init fails return
error code.

fsp_header_init: returns true if header initialization succeeds
and false if not.

fseg_create_general: report error if segment or page allocation fails.

innobase_init: If first datafile is smaller than 3M and could not
contain all doublewrite buffer pages report error and fail to
initialize InnoDB plugin.

row_truncate_table_for_mysql: report error if fsp header init
fails.

srv_init_abort: New function to report database initialization errors.

srv_undo_tablespaces_init, innobase_start_or_create_for_mysql: If
database initialization fails report error and abort.

trx_rseg_create: If segment header creation fails return.
2017-06-01 14:07:48 +03:00
Jan Lindström
50eb40a2a8 MDEV-11738: Mariadb uses 100% of several of my 8 cpus doing nothing
MDEV-11581: Mariadb starts InnoDB encryption threads
when key has not changed or data scrubbing turned off

Background: Key rotation is based on background threads
(innodb-encryption-threads) periodically going through
all tablespaces on fil_system. For each tablespace
current used key version is compared to max key age
(innodb-encryption-rotate-key-age). This process
naturally takes CPU. Similarly, in same time need for
scrubbing is investigated. Currently, key rotation
is fully supported on Amazon AWS key management plugin
only but InnoDB does not have knowledge what key
management plugin is used.

This patch re-purposes innodb-encryption-rotate-key-age=0
to disable key rotation and background data scrubbing.
All new tables are added to special list for key rotation
and key rotation is based on sending a event to
background encryption threads instead of using periodic
checking (i.e. timeout).

fil0fil.cc: Added functions fil_space_acquire_low()
to acquire a tablespace when it could be dropped concurrently.
This function is used from fil_space_acquire() or
fil_space_acquire_silent() that will not print
any messages if we try to acquire space that does not exist.
fil_space_release() to release a acquired tablespace.
fil_space_next() to iterate tablespaces in fil_system
using fil_space_acquire() and fil_space_release().
Similarly, fil_space_keyrotation_next() to iterate new
list fil_system->rotation_list where new tables.
are added if key rotation is disabled.
Removed unnecessary functions fil_get_first_space_safe()
fil_get_next_space_safe()

fil_node_open_file(): After page 0 is read read also
crypt_info if it is not yet read.

btr_scrub_lock_dict_func()
buf_page_check_corrupt()
buf_page_encrypt_before_write()
buf_merge_or_delete_for_page()
lock_print_info_all_transactions()
row_fts_psort_info_init()
row_truncate_table_for_mysql()
row_drop_table_for_mysql()
    Use fil_space_acquire()/release() to access fil_space_t.

buf_page_decrypt_after_read():
    Use fil_space_get_crypt_data() because at this point
    we might not yet have read page 0.

fil0crypt.cc/fil0fil.h: Lot of changes. Pass fil_space_t* directly
to functions needing it and store fil_space_t* to rotation state.
Use fil_space_acquire()/release() when iterating tablespaces
and removed unnecessary is_closing from fil_crypt_t. Use
fil_space_t::is_stopping() to detect when access to
tablespace should be stopped. Removed unnecessary
fil_space_get_crypt_data().

fil_space_create(): Inform key rotation that there could
be something to do if key rotation is disabled and new
table with encryption enabled is created.
Remove unnecessary functions fil_get_first_space_safe()
and fil_get_next_space_safe(). fil_space_acquire()
and fil_space_release() are used instead. Moved
fil_space_get_crypt_data() and fil_space_set_crypt_data()
to fil0crypt.cc.

fsp_header_init(): Acquire fil_space_t*, write crypt_data
and release space.

check_table_options()
	Renamed FIL_SPACE_ENCRYPTION_* TO FIL_ENCRYPTION_*

i_s.cc: Added ROTATING_OR_FLUSHING field to
information_schema.innodb_tablespace_encryption
to show current status of key rotation.
2017-03-14 16:23:10 +02:00
Marko Mäkelä
ad0c218a44 Merge 10.0 into 10.1
Also, implement MDEV-11027 a little differently from 5.5 and 10.0:

recv_apply_hashed_log_recs(): Change the return type back to void
(DB_SUCCESS was always returned).

Report progress also via systemd using sd_notifyf().
2017-03-09 08:53:08 +02:00
Vicențiu Ciorbaru
1acfa942ed Merge branch '5.5' into 10.0 2017-03-03 01:37:54 +02:00
Marko Mäkelä
3c47ed4849 Merge 10.0 into 10.1 2017-02-20 14:02:40 +02:00
Jan Lindström
108b211ee2 Fix gcc 6.3.x compiler warnings.
These are caused by fact that functions are declared with
__attribute__((nonnull)) or left shit like ~0 << macro
when ~0U << macro should be used.
2017-02-16 12:02:31 +02:00
Marko Mäkelä
01d5d6db4c Fix GCC 6.3.0 warnings. 2017-02-16 11:16:27 +02:00
Marko Mäkelä
ab1e6fefd8 MDEV-11623 MariaDB 10.1 fails to start datadir created with
MariaDB 10.0/MySQL 5.6 using innodb-page-size!=16K

The storage format of FSP_SPACE_FLAGS was accidentally broken
already in MariaDB 10.1.0. This fix is bringing the format in
line with other MySQL and MariaDB release series.

Please refer to the comments that were added to fsp0fsp.h
for details.

This is an INCOMPATIBLE CHANGE that affects users of
page_compression and non-default innodb_page_size. Upgrading
to this release will correct the flags in the data files.
If you want to downgrade to earlier MariaDB 10.1.x, please refer
to the test innodb.101_compatibility how to reset the
FSP_SPACE_FLAGS in the files.

NOTE: MariaDB 10.1.0 to 10.1.20 can misinterpret
uncompressed data files with innodb_page_size=4k or 64k as
compressed innodb_page_size=16k files, and then probably fail
when trying to access the pages. See the comments in the
function fsp_flags_convert_from_101() for detailed analysis.

Move PAGE_COMPRESSION to FSP_SPACE_FLAGS bit position 16.
In this way, compressed innodb_page_size=16k tablespaces will not
be mistaken for uncompressed ones by MariaDB 10.1.0 to 10.1.20.

Derive PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR from the
dict_table_t::flags when the table is available, in
fil_space_for_table_exists_in_mem() or fil_open_single_table_tablespace().
During crash recovery, fil_load_single_table_tablespace() will use
innodb_compression_level for the PAGE_COMPRESSION_LEVEL.

FSP_FLAGS_MEM_MASK: A bitmap of the memory-only fil_space_t::flags
that are not to be written to FSP_SPACE_FLAGS. Currently, these will
include PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR.

Introduce the macro FSP_FLAGS_PAGE_SSIZE(). We only support
one innodb_page_size for the whole instance.

When creating a dummy tablespace for the redo log, use
fil_space_t::flags=0. The flags are never written to the redo log files.

Remove many FSP_FLAGS_SET_ macros.

dict_tf_verify_flags(): Remove. This is basically only duplicating
the logic of dict_tf_to_fsp_flags(), used in a debug assertion.

fil_space_t::mark: Remove. This flag was not used for anything.

fil_space_for_table_exists_in_mem(): Remove the unnecessary parameter
mark_space, and add a parameter for table flags. Check that
fil_space_t::flags match the table flags, and adjust the (memory-only)
flags based on the table flags.

fil_node_open_file(): Remove some redundant or unreachable conditions,
do not use stderr for output, and avoid unnecessary server aborts.

fil_user_tablespace_restore_page(): Convert the flags, so that the
correct page_size will be used when restoring a page from the
doublewrite buffer.

fil_space_get_page_compressed(), fsp_flags_is_page_compressed(): Remove.
It suffices to have fil_space_is_page_compressed().

FSP_FLAGS_WIDTH_DATA_DIR, FSP_FLAGS_WIDTH_PAGE_COMPRESSION_LEVEL,
FSP_FLAGS_WIDTH_ATOMIC_WRITES: Remove, because these flags do not
exist in the FSP_SPACE_FLAGS but only in memory.

fsp_flags_try_adjust(): New function, to adjust the FSP_SPACE_FLAGS
in page 0. Called by fil_open_single_table_tablespace(),
fil_space_for_table_exists_in_mem(), innobase_start_or_create_for_mysql()
except if --innodb-read-only is active.

fsp_flags_is_valid(ulint): Reimplement from the scratch, with
accurate comments. Do not display any details of detected
inconsistencies, because the output could be confusing when
dealing with MariaDB 10.1.x data files.

fsp_flags_convert_from_101(ulint): Convert flags from buggy
MariaDB 10.1.x format, or return ULINT_UNDEFINED if the flags
cannot be in MariaDB 10.1.x format.

fsp_flags_match(): Check the flags when probing files.
Implemented based on fsp_flags_is_valid()
and fsp_flags_convert_from_101().

dict_check_tablespaces_and_store_max_id(): Do not access the
page after committing the mini-transaction.

IMPORT TABLESPACE fixes:

AbstractCallback::init(): Convert the flags.

FetchIndexRootPages::operator(): Check that the tablespace flags match the
table flags. Do not attempt to convert tablespace flags to table flags,
because the conversion would necessarily be lossy.

PageConverter::update_header(): Write back the correct flags.
This takes care of the flags in IMPORT TABLESPACE.
2017-01-15 19:05:50 +02:00
Sergei Golubchik
3361aee591 Merge branch '10.0' into 10.1 2016-06-28 22:01:55 +02:00
Sergei Golubchik
a79d46c3a4 Merge branch 'merge-innodb-5.6' into 10.0 2016-06-21 14:58:19 +02:00
Sergei Golubchik
720e04ff67 5.6.31 2016-06-21 14:21:03 +02:00
Sergei Golubchik
a2bcee626d Merge branch '10.0' into 10.1 2015-12-21 21:24:22 +01:00
Sergei Golubchik
5b3c100788 Merge branch 'merge/merge-innodb-5.6' into 10.0 2015-12-13 10:18:42 +01:00
Sergei Golubchik
e9eaaa4b4a 5.6.28 2015-12-13 10:11:49 +01:00
Monty
f813a00029 Fixed failing test cases and compiler warnings found by buildbot
- Added some extra command to rpl_start_stop to ensure that the
  IO thread has connected to the master before we shut down the server.
- if signal returns signalhandler_t, use this with the alarm code
- Added missing tests to sys_vars
- Fixed some possible overflow bugs in tabxml.cpp
2015-11-24 20:04:12 +02:00
Sergei Golubchik
530a6e7481 Merge branch '10.0' into 10.1
referenced_by_foreign_key2(), needed for InnoDB to compile,
was taken from 10.0-galera
2015-09-03 12:58:41 +02:00
Jan Lindström
a9960ef0ec MDEV-8386: MARIADB creates very big tmp file > 351Gb. Started happening after Version 10.0.16-15
Fixed small size limit and added additional information if we reserve
new pages more then 50 times.
2015-07-14 20:14:48 +03:00
Sergei Golubchik
5091a4ba75 Merge tag 'mariadb-10.0.19' into 10.1 2015-06-01 15:51:25 +02:00
Sergei Golubchik
085297a121 5.6.24 2015-05-04 22:13:46 +02:00
Sergei Golubchik
6d06fbbd1d move to storage/innobase 2015-05-04 19:17:21 +02:00
Jan Lindström
b4a4d82396 InnoDB/XtraDB Encryption cleanup.
Step 1:
-- Remove page encryption from dictionary (per table
encryption will be handled by storing crypt_data to page 0)
-- Remove encryption/compression from os0file and all functions
before that (compression will be added to buf0buf.cc)
-- Use same CRYPT_SCHEME_1 for all encryption methods
-- Do some code cleanups to confort InnoDB coding style
2015-04-07 23:44:56 +02:00
Monty
d7d589dc01 Push for testing of encryption 2015-02-10 10:21:17 +01:00
Sergei Golubchik
27d45e4696 MDEV-5574 Set AUTO_INCREMENT below max value of column.
Update InnoDB to 5.6.14
Apply MySQL-5.6 hack for MySQL Bug#16434374
Move Aria-only HA_RTREE_INDEX from my_base.h to maria_def.h (breaks an assert in InnoDB)
Fix InnoDB memory leak
2014-02-01 09:33:26 +01:00
Michael Widenius
068c61978e Temporary commit of 10.0-merge 2013-03-26 00:03:13 +02:00
Michael Widenius
1d0f70c2f8 Temporary commit of merge of MariaDB 10.0-base and MySQL 5.6 2012-08-01 17:27:34 +03:00
Annamalai Gurusami
3ce20ed4f9 Merging from mysql-5.1 to mysql-5.5. 2012-03-09 12:35:27 +05:30
Annamalai Gurusami
29aaa6588d Bug #11766634 59783: InnoDB data grows unexpectedly when inserting,
truncating, inserting the same set of rows. When a table is 
re-created with the same set of rows, the data file size must
not grow.  

rb:968
Approved by Marko.
2012-03-09 11:07:16 +05:30
Marko Mäkelä
4ea57c80b2 Merge mysql-5.1 to mysql-5.5. 2012-02-17 11:52:51 +02:00
unknown
7532976dd9 Bug #13116225 LIVE DOWNGRADE CRASHES WITH INNODB_PAGE_SIZE=4K
This bug ensures that a live downgrade from an InnoDB 5.6 with WL5756 and
a database created with innodb-page-size=8k or 4k will make a version 5.5
installation politely refuse to start. Instead of crashing or giving some
indication about a corrupted database, it will indicate the page size
difference. 

This patch takes only that part of the Wl5756 patch that is needed to
protect against opening a tablespace that is stamped with a different
page size.

It also contains the change in dict_index_find_on_id_low() just in case
a database with another page size is created by recompiling a pre-WL5756
InnoDB.
2011-12-08 13:32:19 -06:00
Marko Mäkelä
91b5e9352a Revert most of revno 3560.9.1 (Bug#12704861)
This was an attempt to address problems with the Bug#12612184 fix.
Even with this follow-up fix, crash recovery can be broken.
Let us fix the bug later.
2011-10-26 11:44:28 +03:00
Inaam Rana
c95fdd936e Revert original fix for Bug 12612184 and the follow up fix for
Bug 12704861.

Bug 12704861 fix was revno: 3504.1.1 (rb://693)
Bug 12612184 fix was revno: 3445.1.10 (rb://678)
2011-09-30 07:02:19 -04:00
Marko Mäkelä
7c20cd1b5d Merge mysql-5.1 to mysql-5.5. 2011-08-29 11:22:43 +03:00
Marko Mäkelä
41f229cd9e Bug#12704861 Corruption after a crash during BLOB update
The fix of Bug#12612184 broke crash recovery. When a record that
contains off-page columns (BLOBs) is updated, we must first write redo
log about the BLOB page writes, and only after that write the redo log
about the B-tree changes. The buggy fix would log the B-tree changes
first, meaning that after recovery, we could end up having a record
that contains a null BLOB pointer.

Because we will be redo logging the writes off the off-page columns
before the B-tree changes, we must make sure that the pages chosen for
the off-page columns are free both before and after the B-tree
changes. In this way, the worst thing that can happen in crash
recovery is that the BLOBs are written to free pages, but the B-tree
changes are not applied. The BLOB pages would correctly remain free in
this case. To achieve this, we must allocate the BLOB pages in the
mini-transaction of the B-tree operation. A further quirk is that BLOB
pages are allocated from the same file segment as leaf pages. Because
of this, we must temporarily "hide" any leaf pages that were freed
during the B-tree operation by "fake allocating" them prior to writing
the BLOBs, and freeing them again before the mtr_commit() of the
B-tree operation, in btr_mark_freed_leaves().

btr_cur_mtr_commit_and_start(): Remove this faulty function that was
introduced in the Bug#12612184 fix. The problem that this function was
trying to address was that when we did mtr_commit() the BLOB writes
before the mtr_commit() of the update, the new BLOB pages could have
overwritten clustered index B-tree leaf pages that were freed during
the update. If recovery applied the redo log of the BLOB writes but
did not see the log of the record update, the index tree would be
corrupted. The correct solution is to make the freed clustered index
pages unavailable to the BLOB allocation. This function is also a
likely culprit of InnoDB hangs that were observed when testing the
Bug#12612184 fix.

btr_mark_freed_leaves(): Mark all freed clustered index leaf pages of
a mini-transaction allocated (nonfree=TRUE) before storing the BLOBs,
or freed (nonfree=FALSE) before committing the mini-transaction.

btr_freed_leaves_validate(): A debug function for checking that all
clustered index leaf pages that have been marked free in the
mini-transaction are consistent (have not been zeroed out).

btr_page_alloc_low(): Refactored from btr_page_alloc(). Return the
number of the allocated page, or FIL_NULL if out of space. Add the
parameter "mtr_t* init_mtr" for specifying the mini-transaction where
the page should be initialized, or if this is a "fake allocation"
(init_mtr=NULL) by btr_mark_freed_leaves(nonfree=TRUE).

btr_page_alloc(): Add the parameter init_mtr, allowing the page to be
initialized and X-latched in a different mini-transaction than the one
that is used for the allocation. Invoke btr_page_alloc_low(). If a
clustered index leaf page was previously freed in mtr, remove it from
the memo of previously freed pages.

btr_page_free(): Assert that the page is a B-tree page and it has been
X-latched by the mini-transaction. If the freed page was a leaf page
of a clustered index, link it by a MTR_MEMO_FREE_CLUST_LEAF marker to
the mini-transaction.

btr_store_big_rec_extern_fields_func(): Add the parameter alloc_mtr,
which is NULL (old behaviour in inserts) and the same as local_mtr in
updates. If alloc_mtr!=NULL, the BLOB pages will be allocated from it
instead of the mini-transaction that is used for writing the BLOBs.

fsp_alloc_from_free_frag(): Refactored from
fsp_alloc_free_page(). Allocate the specified page from a partially
free extent.

fseg_alloc_free_page_low(), fseg_alloc_free_page_general(): Add the
parameter "mtr_t* init_mtr" for specifying the mini-transaction where
the page should be initialized, or NULL if this is a "fake allocation"
that prevents the reuse of a previously freed B-tree page for BLOB
storage. If init_mtr==NULL, try harder to reallocate the specified page
and assert that it succeeded.

fsp_alloc_free_page(): Add the parameter "mtr_t* init_mtr" for
specifying the mini-transaction where the page should be initialized.
Do not allow init_mtr == NULL, because this function is never to be
used for "fake allocations".

mtr_t: Add the operation MTR_MEMO_FREE_CLUST_LEAF and the flag
mtr->freed_clust_leaf for quickly determining if any
MTR_MEMO_FREE_CLUST_LEAF operations have been posted.

row_ins_index_entry_low(): When columns are being made off-page in
insert-by-update, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass
the mini-transaction as the alloc_mtr to
btr_store_big_rec_extern_fields(). Finally, invoke
btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages.

row_build(): Correct a comment, and add a debug assertion that a
record that contains NULL BLOB pointers must be a fresh insert.

row_upd_clust_rec(): When columns are being moved off-page, invoke
btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as
the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke
btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages.

buf_reset_check_index_page_at_flush(): Remove. The function
fsp_init_file_page_low() already sets
bpage->check_index_page_at_flush=FALSE.

There is a known issue in tablespace extension. If the request to
allocate a BLOB page leads to the tablespace being extended, crash
recovery could see BLOB writes to pages that are off the tablespace
file bounds. This should trigger an assertion failure in fil_io() at
crash recovery. The safe thing would be to write redo log about the
tablespace extension to the mini-transaction of the BLOB write, not to
the mini-transaction of the record update. However, there is no redo
log record for file extension in the current redo log format.

rb:693 approved by Sunny Bains
2011-08-29 11:16:42 +03:00
Marko Mäkelä
ee486208cd Merge mysql-5.1-innodb to mysql-5.5-innodb. 2011-02-02 15:58:01 +02:00
Vasil Dimov
a775ea08c8 Merge mysql-5.1-innodb -> mysql-5.5-innodb 2011-01-06 09:19:02 +02:00
Vasil Dimov
15273e4310 (Builtin InnoDB) Fix Bug#59303 Correct URL in crash message
old URL: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
new URL: http://dev.mysql.com/doc/refman/5.1/en/forcing-innodb-recovery.html

Notice that there is a redirect from the old URL to the new URL, so visiting
the old URL does not give "page not found" error.
2011-01-06 09:05:45 +02:00
Sunny Bains
6bcad3dc35 Merge from mysql-5.1-innodb to mysql-5.5-innodb. 2010-11-03 14:14:44 +11:00
Sunny Bains
81da9b3d72 Fix Bug #54538 - use of exclusive innodb dictionary lock limits performance.
This patch doesn't get rid of the need to acquire the dict_sys->mutex but
reduces the need to keep the mutex locked for the duration of the query
to fsp_get_available_space_in_free_extents() from ha_innobase::info().

rb://390.
2010-08-20 17:49:43 +10:00
Marko Mäkelä
c1567ecebd Bug#54728: Replace the dulint struct with a 64-bit integer. 2010-06-23 14:06:59 +03:00
Vasil Dimov
6327a4f0af Merge from mysql-5.1-innodb:
------------------------------------------------------------
  revno: 3441
  revision-id: marko.makela@oracle.com-20100504130917-qmvzbj3pgil2nuat
  parent: marko.makela@oracle.com-20100504125510-gemcfhj7mefrhalo
  committer: Marko M?kel? <marko.makela@oracle.com>
  branch nick: 5.1-innodb
  timestamp: Tue 2010-05-04 16:09:17 +0300
  message:
    fsp_init_file_page_low(): Zero out the page. (Bug #53306)
  modified:
    storage/innodb_plugin/fsp/fsp0fsp.c 2@16c675df-0fcb-4bc9-8058-dcc011a37293:trunk%2Ffsp%2Ffsp0fsp.c
  ------------------------------------------------------------
2010-05-04 16:57:09 +03:00
Marko Mäkelä
304daa7431 fsp_init_file_page_low(): Zero out the page. (Bug #53306) 2010-05-04 16:13:58 +03:00
Vasil Dimov
c877ff39bc Import branches/innodb+ from SVN on top of storage/innobase. 2010-04-12 18:20:41 +03:00
Satya B
d63eb541f5 Merging Innodb plugin 1.0.5 revisions from 5.1-main from revisions 3149 to 3163
also merged missing Innodb plugin revisions r5636,r5635 manually
2009-10-16 17:28:02 +05:30
Guilhem Bichot
7ceb29ff17 Renamed storage/innodb_plugin to storage/innobase, so that 1) it's the same
layout as we always had in trees containing only the builtin
2) win\configure.js WITH_INNOBASE_STORAGE_ENGINE still works.

storage/innobase/CMakeLists.txt:
  fix to new directory name (and like 5.1)
storage/innobase/Makefile.am:
  fix to new directory name (and like 5.1)
storage/innobase/handler/ha_innodb.cc:
  fix to new directory name (and like 5.1)
storage/innobase/plug.in:
  fix to new directory name (and like 5.1)
2009-08-07 12:16:00 +02:00
Guilhem Bichot
b57e4dbd88 Creation of mysql-trunk = {summit + "Innodb plugin replacing the builtin"}:
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin 
bzr rm innobase # remove the builtin
Next step: build, test fixes.
2009-08-04 13:25:19 +02:00
unknown
c2f0c211cd Apply snapshot innodb-51-ss1644
Fixes:
- Bug #23710: crash_commit_before fails if innodb_file_per_table=1
- Bug #28254: innodb crash if shutdown during innodb_table_monitor is running
- Bug #28604: innodb_force_recovery restricts data dump
- Bug #29097: fsp_get_available_space_in_free_extents() is capped at 4TB
- Bug #29155: Innodb "Parallel recovery" is not prevented


storage/innobase/Makefile.am:
  Apply snapshot innodb-51-ss1644
  
  Revision r1632:
  Add include/lock0priv.h to noinst_HEADERS in Makefile.am.
  
  
  
  Revision r1636:
  Move lock_get_type() from lock/lock0lock.c to include/lock0priv.ic:
  * lock0lock.c: remove lock_get_type() and include include/lock0priv.ic
  * lock0priv.h: include lock0priv.ic and add lock_get_type() prototype
  * Makefile.am: add lock0priv.ic to noinst_HEADERS
  * lock0priv.ic: introduce this new file containing the body of
    lock_get_type()
  
  This move is necessary in order to use lock_get_type() from other lock/
  source files (it's going to be used in lock/lock0iter.c).
  
  Approved by:	Heikki
  
  
  Revision r1638:
  Introduce a lock queue iterator for easy (and opaque) traversing of lock
  queues. Supports table and record lock queues via the same interface.
  
  There is only "get previous" method because currently there is no need
  for "get next" - it would be unused. Feel free to add one if needed.
  
  Approved by:	Heikki
storage/innobase/buf/buf0buf.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1624:
  Fix change missed as part of Bug 15815. Use a function to check if a block
  needs to be made younger.
  
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/buf/buf0lru.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/fsp/fsp0fsp.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1605:
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
storage/innobase/handler/ha_innodb.cc:
  Apply snapshot innodb-51-ss1644
  
  Revision r1605:
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
  
  
  Revision r1573:
  create_table_def(): Eliminate the inline function call to dict_table_is_comp()
  that was introduced in r1571.  Inlining is disabled in ha_innodb.cc.
  
  
  Revision r1574:
  innodb_check_for_record_too_big_error(): Divide the return value of
  page_get_free_space_of_empty_noninline() by 2.  Until r1571, that function
  did not return the same value as page_get_free_space_of_empty().
  
  
  Revision r1571:
  Fix a severe bug that was introduced in r1422 when fixing Bug 21101.
  When creating an index containing a too long record, InnoDB would
  dereference a NULL pointer when trying to determine the maximum row length.
  
  innodb_check_for_record_too_big_error(): Replace the dict_table_t*
  parameter with a Boolean flag.  There is not always a dict_table_t object
  when this function is called.
  
  page_get_free_space_of_empty_noninline(): Move the definition and
  declaration from row0mysql (!) to page0page.  Make the signature
  identical with page_get_free_space_of_empty().
  
  create_clustered_index_when_no_primary(): Add the parameter "comp".
  Remove unnecessary casts.
storage/innobase/include/buf0buf.ic:
  Apply snapshot innodb-51-ss1644
  
  Revision r1624:
  Fix change missed as part of Bug 15815. Use a function to check if a block
  needs to be made younger.
storage/innobase/include/fsp0fsp.h:
  Apply snapshot innodb-51-ss1644
  
  Revision r1605:
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
storage/innobase/include/lock0lock.h:
  Apply snapshot innodb-51-ss1644
  
  Revision r1623:
  Fix typo in comment.
  
  
  Revision r1628:
  lock_has_to_wait() is needed in the INFORMATION_SCHEMA implementation
  in order to determine which lock is blocking which. Make it non-static
  and put its definition in include/lock0lock.h.
  
  Approved by:	Heikki (via IM)
storage/innobase/include/mem0mem.ic:
  Apply snapshot innodb-51-ss1644
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/include/page0page.h:
  Apply snapshot innodb-51-ss1644
  
  Revision r1571:
  Fix a severe bug that was introduced in r1422 when fixing Bug 21101.
  When creating an index containing a too long record, InnoDB would
  dereference a NULL pointer when trying to determine the maximum row length.
  
  innodb_check_for_record_too_big_error(): Replace the dict_table_t*
  parameter with a Boolean flag.  There is not always a dict_table_t object
  when this function is called.
  
  page_get_free_space_of_empty_noninline(): Move the definition and
  declaration from row0mysql (!) to page0page.  Make the signature
  identical with page_get_free_space_of_empty().
  
  create_clustered_index_when_no_primary(): Add the parameter "comp".
  Remove unnecessary casts.
storage/innobase/include/row0mysql.h:
  Apply snapshot innodb-51-ss1644
  
  Revision r1571:
  Fix a severe bug that was introduced in r1422 when fixing Bug 21101.
  When creating an index containing a too long record, InnoDB would
  dereference a NULL pointer when trying to determine the maximum row length.
  
  innodb_check_for_record_too_big_error(): Replace the dict_table_t*
  parameter with a Boolean flag.  There is not always a dict_table_t object
  when this function is called.
  
  page_get_free_space_of_empty_noninline(): Move the definition and
  declaration from row0mysql (!) to page0page.  Make the signature
  identical with page_get_free_space_of_empty().
  
  create_clustered_index_when_no_primary(): Add the parameter "comp".
  Remove unnecessary casts.
storage/innobase/include/univ.i:
  Apply snapshot innodb-51-ss1644
  
  Revision r1605:
  Fix Bug#29097 "fsp_get_available_space_in_free_extents() is capped at 4TB"
  by typecasting the variables before multiplying them, so that the result of
  the multiplication is of type "unsigned long long".
  
  I verified this fix by creating a sparse file of 6TB and forcing InnoDB to
  use it without overwriting it with zeroes (by commenting the code that
  overwrites :newraw files).
  
  New type ullint is introduced with the sole purpose of shortening
  "unsigned long long", please do not define it to something else than
  "unsigned long long".
  
  Approved by:	Heikki
  
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/lock/lock0lock.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1631:
  Move lock_rec_find_set_bit() and lock_rec_get_prev() from
  lock/lock0lock.c to include/lock0priv.h and make them non-static.
  They will be used in lock/lock0iter.c.
  
  Approved by:	Heikki
  
  
  Revision r1636:
  Move lock_get_type() from lock/lock0lock.c to include/lock0priv.ic:
  * lock0lock.c: remove lock_get_type() and include include/lock0priv.ic
  * lock0priv.h: include lock0priv.ic and add lock_get_type() prototype
  * Makefile.am: add lock0priv.ic to noinst_HEADERS
  * lock0priv.ic: introduce this new file containing the body of
    lock_get_type()
  
  This move is necessary in order to use lock_get_type() from other lock/
  source files (it's going to be used in lock/lock0iter.c).
  
  Approved by:	Heikki
  
  
  Revision r1628:
  lock_has_to_wait() is needed in the INFORMATION_SCHEMA implementation
  in order to determine which lock is blocking which. Make it non-static
  and put its definition in include/lock0lock.h.
  
  Approved by:	Heikki (via IM)
  
  
  Revision r1629:
  Add "const" qualifiers to lock_get_type() and lock_get_mode().
  
  Approved by:	Sunny
  
  
  Revision r1626:
  Move lock_*struct structures from lock/lock0lock.c to include/lock0priv.h.
  
  This is needed in order to add more code to lock/ that uses members of
  these structures (internal to the lock module) but in a separate file,
  rather than lock0lock.c. lock0lock.c is a way too big already.
  
  Approved by:	Sunny
storage/innobase/log/log0recv.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1607:
  Bug#23710
  
  At InnoDB startup consider the case where log scan went beyond checkpoint_lsn as a crash and initiate crash recovery code path.
  
  reviewed by: Heikki
storage/innobase/mem/mem0mem.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/mem/mem0pool.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/os/os0file.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1613:
  Fix Bug#29155 by enabling file locking on FreeBSD.
  It has been disabled because InnoDB has refused to start on
  FreeBSD & LinuxThreads, but now it starts just fine.
  
  Approved by:	Heikki
storage/innobase/page/page0page.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1571:
  Fix a severe bug that was introduced in r1422 when fixing Bug 21101.
  When creating an index containing a too long record, InnoDB would
  dereference a NULL pointer when trying to determine the maximum row length.
  
  innodb_check_for_record_too_big_error(): Replace the dict_table_t*
  parameter with a Boolean flag.  There is not always a dict_table_t object
  when this function is called.
  
  page_get_free_space_of_empty_noninline(): Move the definition and
  declaration from row0mysql (!) to page0page.  Make the signature
  identical with page_get_free_space_of_empty().
  
  create_clustered_index_when_no_primary(): Add the parameter "comp".
  Remove unnecessary casts.
storage/innobase/rem/rem0rec.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/row/row0mysql.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1571:
  Fix a severe bug that was introduced in r1422 when fixing Bug 21101.
  When creating an index containing a too long record, InnoDB would
  dereference a NULL pointer when trying to determine the maximum row length.
  
  innodb_check_for_record_too_big_error(): Replace the dict_table_t*
  parameter with a Boolean flag.  There is not always a dict_table_t object
  when this function is called.
  
  page_get_free_space_of_empty_noninline(): Move the definition and
  declaration from row0mysql (!) to page0page.  Make the signature
  identical with page_get_free_space_of_empty().
  
  create_clustered_index_when_no_primary(): Add the parameter "comp".
  Remove unnecessary casts.
storage/innobase/sync/sync0rw.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1598:
  Add some comments.
  
  Approved by:	Heikki (via IM)
storage/innobase/sync/sync0sync.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1598:
  Add some comments.
  
  Approved by:	Heikki (via IM)
storage/innobase/trx/trx0sys.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
storage/innobase/trx/trx0trx.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1595:
  trx_commit_for_mysql(): Avoid acquiring and releasing kernel_mutex when
  trx->sess or trx_dummy_sess is non-NULL.
storage/innobase/ut/ut0mem.c:
  Apply snapshot innodb-51-ss1644
  
  Revision r1581:
  Port extra Valgrind instrumentation (UNIV_DEBUG_VALGRIND) from branches/zip.
2007-07-24 19:34:31 -06:00