Commit graph

5025 commits

Author SHA1 Message Date
Inaam Rana
01748ce128 Bug #14100254 65389: MVCC IS BROKEN WITH IMPLICIT LOCK
rb://1088
approved by: Marko Makela

This bug was introduced in early stages of plugin. We were not
checking for an implicit lock on sec index rec for trx_id that is
stamped on current version of the clust_index in case where the
clust_index has a previous delete marked version.
2012-05-24 12:37:03 -04:00
Annamalai Gurusami
ce9e6b5a9a Bug #13943231: ALTER TABLE AFTER DISCARD MAY CRASH THE SERVER
The following scenario crashes our mysql server:

1.  set global innodb_file_per_table=1;
2.  create table t1(c1 int) engine=innodb;
3.  alter table t1 discard tablespace;
4.  alter table t1 add unique index(c1);

Step 4 crashes the server.  This patch introduces a check on discarded
tablespace to avoid the crash.

rb://1041 approved by Marko Makela
2012-05-16 16:36:49 +05:30
Venkata Sidagam
6b05e434bf Bug #13955256: KEYCACHE CRASHES, CORRUPTIONS/HANGS WITH,
FULLTEXT INDEX AND CONCURRENT DML.

Problem Statement:
------------------
1) Create a table with FT index.
2) Enable concurrent inserts.
3) In multiple threads do below operations repeatedly
   a) truncate table
   b) insert into table ....
   c) select ... match .. against .. non-boolean/boolean mode

After some time we could observe two different assert core dumps

Analysis:
--------
1)assert core dump at key_read_cache():
Two select threads operating in-parallel on same key 
root block.
1st select thread block->status is set to BLOCK_ERROR 
because the my_pread() in read_block() is returning '0'. 
Truncate table made the index file size as 1024 and pread 
was asked to get the block of count bytes(1024 bytes) 
from offset of 1024 which it cannot read since its 
"end of file" and retuning '0' setting 
"my_errno= HA_ERR_FILE_TOO_SHORT" and the key_file_length, 
key_root[0] is same i.e. 1024. Since block status has BLOCK_ERROR 
the 1st select thread enter into the free_block() and will 
be under wait on conditional mutex by making status as 
BLOCK_REASSIGNED and goes for wait_on_readers(). Other select 
thread will also work on the same block and sees the status as 
BLOCK_ERROR and enters into free_block(), checks for BLOCK_REASSIGNED 
and asserting the server.

2)assert core dump at key_write_cache():
One select thread and One insert thread.
Select thread gets the unlocks the 'keycache->cache_lock', 
which allows other threads to continue and gets the pread() 
return value as'0'(please see the explanation above) and 
tries to get the lock on 'keycache->cache_lock' and waits 
there for the lock.
Insert thread requests for the block, block will be assigned 
from the hash list and makes the page_status as 
'PAGE_WAIT_TO_BE_READ' and goes for the read_block(), waits 
in the queue since there are some other threads performing 
reads on the same block.
Select thread which was waiting for the 'keycache->cache_lock' 
mutex in the read_block() will continue after getting the my_pread() 
value as '0' and sets the block status as BLOCK_ERROR and goes to 
the free_block() and go to the wait_for_readers().
Now the insert thread will awake and continues. and checks 
block->status as not BLOCK_READ and it asserts.  

Fix:
---
In the full text code, multiple readers of index file is not guarded. 
Hence added below below code in _ft2_search() and walk_and_match().

to lock the key_root I have used below code in _ft2_search()
 if (info->s->concurrent_insert)
    mysql_rwlock_rdlock(&share->key_root_lock[0]);

and to unlock 
 if (info->s->concurrent_insert)
   mysql_rwlock_unlock(&share->key_root_lock[0]);

storage/myisam/ft_boolean_search.c:
  Since its a recursion function, to avoid confusion in taking and 
  releasing the locks, renamed _ft2_search() to _ft2_search_internal() 
  function. And _ft2_search() will take the lock, call 
  _ft2_search_internal() and release the lock in case of concurrent 
  inserts.
storage/myisam/ft_nlq_search.c:
  Added read locks code in walk_and_match()
2012-05-16 16:14:27 +05:30
Marko Mäkelä
5917c58a5d Bug#14025221 FOREIGN KEY REFERENCES FREED MEMORY AFTER DROP INDEX
dict_table_replace_index_in_foreign_list(): Replace the dropped index
also in the foreign key constraints of child tables that are
referencing this table.

row_ins_check_foreign_constraint(): If the underlying index is
missing, refuse the operation.

rb:1051 approved by Jimmy Yang
2012-05-15 15:04:39 +03:00
Annamalai Gurusami
31a6bcbee4 Merge from mysql-5.1 to mysql-5.5. 2012-05-16 16:33:22 +05:30
Venkata Sidagam
7244e84e6d Merging the fix from mysql-5.1 to mysql-5.5 2012-05-16 16:06:07 +05:30
Annamalai Gurusami
7a6c449c62 Bug #13943231: ALTER TABLE AFTER DISCARD MAY CRASH THE SERVER
The following scenario crashes our mysql server:

1.  set global innodb_file_per_table=1;
2.  create table t1(c1 int) engine=innodb;
3.  alter table t1 discard tablespace;
4.  alter table t1 add unique index(c1);

Step 4 crashes the server.  This patch introduces a check on discarded
tablespace to avoid the crash.

rb://1041 approved by Marko Makela
2012-05-16 15:23:56 +05:30
Venkata Sidagam
0e9c1e9fc3 Bug #13955256: KEYCACHE CRASHES, CORRUPTIONS/HANGS WITH,
FULLTEXT INDEX AND CONCURRENT DML.

Problem Statement:
------------------
1) Create a table with FT index.
2) Enable concurrent inserts.
3) In multiple threads do below operations repeatedly
   a) truncate table
   b) insert into table ....
   c) select ... match .. against .. non-boolean/boolean mode

After some time we could observe two different assert core dumps

Analysis:
--------
1)assert core dump at key_read_cache():
Two select threads operating in-parallel on same key 
root block.
1st select thread block->status is set to BLOCK_ERROR 
because the my_pread() in read_block() is returning '0'. 
Truncate table made the index file size as 1024 and pread 
was asked to get the block of count bytes(1024 bytes) 
from offset of 1024 which it cannot read since its 
"end of file" and retuning '0' setting 
"my_errno= HA_ERR_FILE_TOO_SHORT" and the key_file_length, 
key_root[0] is same i.e. 1024. Since block status has BLOCK_ERROR 
the 1st select thread enter into the free_block() and will 
be under wait on conditional mutex by making status as 
BLOCK_REASSIGNED and goes for wait_on_readers(). Other select 
thread will also work on the same block and sees the status as 
BLOCK_ERROR and enters into free_block(), checks for BLOCK_REASSIGNED 
and asserting the server.

2)assert core dump at key_write_cache():
One select thread and One insert thread.
Select thread gets the unlocks the 'keycache->cache_lock', 
which allows other threads to continue and gets the pread() 
return value as'0'(please see the explanation above) and 
tries to get the lock on 'keycache->cache_lock' and waits 
there for the lock.
Insert thread requests for the block, block will be assigned 
from the hash list and makes the page_status as 
'PAGE_WAIT_TO_BE_READ' and goes for the read_block(), waits 
in the queue since there are some other threads performing 
reads on the same block.
Select thread which was waiting for the 'keycache->cache_lock' 
mutex in the read_block() will continue after getting the my_pread() 
value as '0' and sets the block status as BLOCK_ERROR and goes to 
the free_block() and go to the wait_for_readers().
Now the insert thread will awake and continues. and checks 
block->status as not BLOCK_READ and it asserts.  

Fix:
---
In the full text code, multiple readers of index file is not guarded. 
Hence added below below code in _ft2_search() and walk_and_match().

to lock the key_root I have used below code in _ft2_search()
 if (info->s->concurrent_insert)
    mysql_rwlock_rdlock(&share->key_root_lock[0]);

and to unlock 
 if (info->s->concurrent_insert)
   mysql_rwlock_unlock(&share->key_root_lock[0]);

storage/myisam/ft_boolean_search.c:
  Since its a recursion function, to avoid confusion in taking and 
  releasing the locks, renamed _ft2_search() to _ft2_search_internal() 
  function. And _ft2_search() will take the lock, call 
  _ft2_search_internal() and release the lock in case of concurrent 
  inserts.
storage/myisam/ft_nlq_search.c:
  Added read locks code in walk_and_match()
2012-05-16 13:55:22 +05:30
Marko Mäkelä
a6da754a31 Merge mysql-5.1 to mysql-5.5. 2012-05-15 15:11:34 +03:00
Annamalai Gurusami
326b40c9c8 Merging from mysql-5.1 to mysql-5.5. 2012-05-10 10:33:16 +05:30
Annamalai Gurusami
391ea219c2 Bug #14007649 65111: INNODB SOMETIMES FAILS TO UPDATE ROWS INSERTED
BY A CONCURRENT TRANSACTIO

The member function QUICK_RANGE_SELECT::init_ror_merged_scan() performs
a table handler clone. Innodb does not provide a clone operation.  
The ha_innobase::clone() is not there. The handler::clone() does not 
take care of the ha_innobase->prebuilt->select_lock_type.  Because of 
this what happens is that for one index we do a locking read, and 
for the other index we were doing a non-locking (consistent) read. 
The patch introduces ha_innobase::clone() member function.  
It is implemented similar to ha_myisam::clone().  It calls the 
base class handler::clone() and then does any additional operation 
required.  I am setting the ha_innobase->prebuilt->select_lock_type 
correctly. 

rb://1060 approved by Marko
2012-05-10 10:18:31 +05:30
Sunanda Menon
074ce71e90 Merge from mysql-5.1.63-release 2012-05-08 07:19:14 +02:00
Joerg Bruehe
5be07ceadd Merge 5.5.24 back into main 5.5.
This is a weave merge, but without any conflicts.
In 14 source files, the copyright year needed to be updated to 2012.
2012-05-07 22:20:42 +02:00
Yasufumi Kinoshita
96b62513f6 Bug#11758510 (#50723): INNODB CHECK TABLE FATAL SEMAPHORE WAIT TIMEOUT POSSIBLY TOO SHORT FOR BI
Fixed not to check timeout during the check table.
2012-04-27 19:40:12 +09:00
Yasufumi Kinoshita
08e7444e54 Bug#11758510 (#50723): INNODB CHECK TABLE FATAL SEMAPHORE WAIT TIMEOUT POSSIBLY TOO SHORT FOR BI
Fixed not to check timeout during the check table.
2012-04-27 19:38:13 +09:00
irana
02522904bc merge from 5.1 2012-04-26 08:21:25 -07:00
irana
1a2cf649dc InnoDB: Adjust error message when a dropped tablespace is accessed. 2012-04-26 08:17:14 -07:00
Inaam Rana
08ead005b1 Bug#13990648: 65061: LRU FLUSH RATE CALCULATION IS BASED ON INVALID VALUES
rb://1043
approved by: Sunny Bains

Two internal counters were incremented twice for a single
operations. The counters are:
srv_buf_pool_flushed
buf_lru_flush_page_count
2012-04-23 22:15:29 -04:00
Inaam Rana
2415d955c8 Bug#12677594 - 61575: INNODB: WARNING: IO_SETUP() FAILED WITH EAGAIN.
rb://1033
approved by: Marko Makela

Check return value from os_aio_init() and refuse to start if it fails.
2012-04-23 06:39:16 -04:00
Georgi Kodinov
a84ca72224 merge mysql-5.5->mysql-5.5-security 2012-04-12 14:04:12 +03:00
unknown
99b18a036c Bug#11815557 60269: MYSQL SHOULD REJECT ATTEMPTS TO CREATE SYSTEM
TABLES IN INCORRECT ENGINE

PROBLEM:
  CREATE/ALTER TABLE currently can move system tables like
mysql.db, user, host etc, to engines other than MyISAM. This is not
completely supported as of now, by mysqld. When some of system tables
like plugin, servers, event, func, *_priv, time_zone* are moved
to innodb, mysqld restart crashes. Currently system tables
can be moved to BLACKHOLE also!!!.

ANALYSIS:
  The problem is that there is no check before creating or moving
a system table to some particular engine.

  System tables are suppose to be residing in MyISAM. We can think
of restricting system tables to exist only in MyISAM. But, there could
be future needs of these system tables to be part of other engines
by design. For eg, NDB cluster expects some tables to be on innodb
or ndb engine. This calls for a solution, by which system
tables can be supported by any desired engine, with minimal effort.

FIX:
  The solution provides a handlerton interface using which,
mysqld server can query particular storage engine handlerton for
system tables that it supports. This way each storage engine
layer can define their own system database and system tables.

  The check_engine() function uses the new handlerton function
ha_check_if_supported_system_table() to check if db.tablename
provided in the DDL is supported by the SE.

Note: This fix has modified a test in help.test, which was moving
mysql.help_* to innodb. The primary intention of the test was not
to move them between engines.
2012-04-11 15:53:17 +05:30
Georgi Kodinov
7fa28bcf56 merge mysql-5.5->mysql-5.5-security 2012-04-10 14:23:17 +03:00
Georgi Kodinov
df905524b2 merge mysql-5.1->mysql-5.1-security 2012-04-10 14:21:57 +03:00
Sergey Glukhov
ea1b0492a0 5.1-security -> 5.5-security merge 2012-04-04 14:19:00 +04:00
Sergey Glukhov
b5c690aa54 Bug#11766300 59387: FAILING ASSERTION: CURSOR->POS_STATE == 1997660512 (BTR_PCUR_IS_POSITIONE
Bug#13639204 64111: CRASH ON SELECT SUBQUERY WITH NON UNIQUE INDEX
The crash happened due to wrong calculation
of key length during creation of reference for
sort order index. The problem is that
keyuse->used_tables can have OUTER_REF_TABLE_BIT enabled
but used_tables parameter(create_ref_for_key() func) does
not have it. So key parts which have OUTER_REF_TABLE_BIT
are ommited and it could lead to incorrect key length
calculation(zero key length).


mysql-test/r/subselect_innodb.result:
  test result
mysql-test/t/subselect_innodb.test:
  test case
sql/sql_select.cc:
  added OUTER_REF_TABLE_BIT to the used_tables parameter
  for create_ref_for_key() function.
storage/innobase/handler/ha_innodb.cc:
  added assertion, request from Inno team
storage/innodb_plugin/handler/ha_innodb.cc:
  added assertion, request from Inno team
2012-04-04 13:29:45 +04:00
Inaam Rana
96e9dd03ea Fix Sun Studio compiler warnings due to extra ',' introduced in
rev_id: inaam.rana@oracle.com-20120326134501-23554a8vhjg0wqbm
2012-04-03 09:45:10 -04:00
Sunny Bains
d1809a5c85 Bug #13817703 - auto_increment_offset != 1 + innodb_autoinc_lock_mode=1 => bulk inserts fail
Fix the calculation of the next autoinc value when offset > 1. Some of the
results have changed due to the changes in the allocation calculation. The
new calculation will result in slightly bigger gaps for bulk inserts.
  
rb://866 Approved by Jimmy Yang.
Backported from mysql-trunk (5.6)
2012-03-29 18:02:08 +11:00
Marc Alff
7553f9eefd Bug#13898343 THREAD LOOPS ENDLESSLY IN LF_PINBOX_PUT_PINS WHILE HOLDING
LOCK_THREAD_COUNT

When using the performance schema file io instrumentation in MySQL 5.5,
a thread would loop forever inside lf_pinbox_put_pins, when disconnecting.
It would also hold LOCK_thread_count while doing so, effectively killing the
server.

The root cause of the loop in lf_pinbox_put_pins() is a leak of LF_PINS,
when used with the filename_hash LF_HASH table in the performance schema.

This fix contains the following changes:

1)
Added the missing call to lf_hash_search_unpin(), to prevent the leak.

2)
In mysys/lf_alloc-pin.c, there was some extra debugging code
(MY_LF_EXTRA_DEBUG) written to detect precisely this kind of issues,
but it was never used.
Replaced MY_LF_EXTRA_DEBUG with DBUG_OFF, so that leaks similar to this one
can be always detected in regular debug builds.

3)
Backported the fix for the following bug, from 5.6 to 5.5:
Bug#13417446 - 63339: INCORRECT FILE PATH IN PEFORMANCE_SCHEMA ON WINDOWS
2012-03-28 17:54:30 +02:00
Gleb Shchepa
46d848b8e1 Bug #11880012: INDEX_SUBQUERY, BLACKHOLE,
HANG IN PREPARING WITH 100% CPU USAGE

Infinite loop in the subselect_indexsubquery_engine::exec()
function caused Server hang with 100% CPU usage.

The BLACKHOLE storage engine didn't update handler's
table->status variable after index operations, that
caused an infinite "while(!table->status)" execution.

Index access methods of the BLACKHOLE engine handler
have been updated to set table->status variable to
STATUS_NOT_FOUND or 0 when such a method returns a
HA_ERR_END_OF_FILE error or 0 respectively.


mysql-test/r/blackhole.result:
  Bug #11880012: INDEX_SUBQUERY, BLACKHOLE,
                 HANG IN PREPARING WITH 100% CPU USAGE
  
  New test file for the BLACKHOLE engine.
mysql-test/t/blackhole.test:
  Bug #11880012: INDEX_SUBQUERY, BLACKHOLE,
                 HANG IN PREPARING WITH 100% CPU USAGE
  
  New test file for the BLACKHOLE engine.
storage/blackhole/ha_blackhole.cc:
  Bug #11880012: INDEX_SUBQUERY, BLACKHOLE,
                 HANG IN PREPARING WITH 100% CPU USAGE
  
  Index access methods of the BLACKHOLE engine handler
  have been updated to set table->status variable to
  STATUS_NOT_FOUND or 0 when such a method returns a
  HA_ERR_END_OF_FILE error or 0 respectively, like
  we do in MyISAM storage engine.
2012-03-28 12:22:31 +04:00
Sunny Bains
1e10b60629 Merge from mysql-5.1. 2012-03-28 13:59:06 +11:00
Sunny Bains
83a5a20d81 Merge from mysql-5.0 2012-03-28 13:35:44 +11:00
Inaam Rana
5352a76134 Bug#13704145: ELIMINATE LRU SCAN WHEN DROPPING A TABLE
rb://942
approved by: Marko Makela

We don't need to scan LRU for dropping AHI entries when DROPing a table.
AHI entries are already removed when we free up extents for the btree.
2012-03-26 09:45:01 -04:00
Jimmy Yang
19d68d68df Fix Bug #13849910 - INNODB ASSERTS ON TOO LONG TABLENAME WHEN USING PARTITIONS
rb://981 approved by Sunny Bains
2012-03-22 11:22:54 +08:00
Georgi Kodinov
5bf1a28915 merge mysql-5.5->mysql-5.5-security 2012-03-21 14:56:29 +02:00
Georgi Kodinov
b9b6548966 merge mysql-5.1->mysql-5.1-security 2012-03-21 14:53:09 +02:00
Jimmy Yang
faa3ecdb07 backport Bug #47707 print some progress messages during shutdown of innodb
to mysql-5.5

rb://979 approved by Marko
2012-03-21 11:48:12 +08:00
Joerg Bruehe
747fbf8f8b Merge the 5.5.22 release build into main 5.5,
conflict in "sql/filesort.cc" solved manually.
2012-03-20 22:27:49 +01:00
unknown
3e623252c7 Merge from mysql-5.1.62-release 2012-03-20 17:35:41 +01:00
Jimmy Yang
7f4d66d4cf Fix Bug #13860722 - INCOMPLETE PERFORMANCE SCHEMA INSTRUMENTATION IN INNODB 2012-03-20 17:44:51 +08:00
Inaam Rana
3c1bdb356d merge from 5.1 2012-03-15 13:34:50 -04:00
Inaam Rana
0023147055 Bug#13825266 RACE IN LOCK_VALIDATE() WHEN ACCESSING PAGES DIRECTLY
FROM BUFFER POOL

rb://975
approved by: Marko Makela

There is a race in lock_validate() where we try to access a page
without ensuring that the tablespace stays valid during the operation
i.e.: it is not deleted. This patch tries to fix that by using an
existing flag (the flag is renamed to make it's name more generic
in line with it's new use).
2012-03-15 13:30:17 -04:00
Inaam Rana
1ca73ab435 merge from 5.1 2012-03-15 12:45:24 -04:00
Inaam Rana
04c96834cc Bug#13851171 STRING OVERFLOW IN INNODB CODE FOUND BY STATIC ANALYSIS
rb://976
approved by: Marko Makela

Add an assertion to ensure that string overflow is not happening.
Pointed by Coverity analysis.
2012-03-15 12:38:40 -04:00
Inaam Rana
2ceb117750 merge from 5.1 2012-03-15 11:57:01 -04:00
Inaam Rana
a8217b7cd3 Bug#13537504 VALGRIND: COND. JUMP/MOVE DEPENDS ON UNINITIALISED VALUES
IN OS_THREAD_EQ

rb://977
approved by: Marko Makela

rw_lock::writer_thread field contains the thread id of current x-holder
or wait-x thread. This field is un-initialized at lock creation and is
written to for the first time when an attempt is made to x-lock.

Current code considers ::writer_thread as valid memory region only when
the lock is held in x-mode (or there is an x-waiter). This is an
overkill and it generates valgrind warnings.

The fix is to consider ::writer_thread as valid memory region once it
has been written to.

Reasoning:
==========
The ::writer_thread can be safely considered valid because:

* We only ever do comparison with current calling threads id.
* We only ever do comparison when ::recursive flag is set
* We always unset ::recursive flag in x-unlock
* Same thread cannot be unlocking and attempting to lock at the same
time
* thread_id recycling is not an issue because before an id is recycled
the thread must leave innodb meaning it must release all locks meaning
it must unset ::recursive flag.
2012-03-15 11:53:30 -04:00
Jon Olav Hauglid
29443d085c Merge from mysql-5.5 to mysql-5.5-security
No conflicts
2012-03-15 11:25:53 +01:00
Inaam Rana
e56854d8e5 Bug#13789853 SHOW ENGINE INNODB STATUS HANGS DUE TO EXCESSIVE WORK
IN LOCK_VALIDATE()

rb://917
approved by: Marko Makela

In lock_validate() the limit is used to release the kernel_mutex during
the validation, to obey the latching order.
If we do the limit++ then we are rechecking the same lock most times on
each iteration because limit is being incremented by one and
<space, page_no> will nearly always be > limit. If we set the limit
correctly to (space, page+1) then we are actually making progress
during the iteration.
2012-03-12 13:04:54 -04:00
Annamalai Gurusami
29aaa6588d Bug #11766634 59783: InnoDB data grows unexpectedly when inserting,
truncating, inserting the same set of rows. When a table is 
re-created with the same set of rows, the data file size must
not grow.  

rb:968
Approved by Marko.
2012-03-09 11:07:16 +05:30
Annamalai Gurusami
3ce20ed4f9 Merging from mysql-5.1 to mysql-5.5. 2012-03-09 12:35:27 +05:30
Georgi Kodinov
9c366f7921 merge mysql-5.5->mysql-5.5-security 2012-03-08 17:19:54 +02:00
Georgi Kodinov
4b5306fd6e merge mysql-5.1->mysql-5.1-security 2012-03-08 17:16:53 +02:00
Marko Mäkelä
26ab3d1258 Merge mysql-5.1 to mysql-5.5. 2012-03-08 17:12:42 +02:00
Marko Mäkelä
6e6ba8654b Fix a compiler warning about possibly uninitiaizlied variable. 2012-03-08 17:10:10 +02:00
Marko Mäkelä
9747c33b31 Merge mysql-5.1 to mysql-5.5. 2012-03-08 15:03:40 +02:00
Marko Mäkelä
c5511bdf08 Bug#13807811 BTR_PCUR_RESTORE_POSITION() CAN SKIP A RECORD
This bug has been there at least since MySQL 4.0.9. (Before 4.0.9, the
code probably was even more severely broken.)

btr_pcur_restore_position(): When cursor restoration fails, before
invoking btr_pcur_store_position() move to the previous or next record
unless cursor->rel_pos==BTR_PCUR_ON or the record was not a user
record.

This bug can cause skipped records when btr_pcur_store_position() is
called on the last record of a page. A symptom would be record count
mismatch in CHECK TABLE, or failure to find a record to delete-mark or
update or purge. The following operations should be affected by the
bug:

* row_search_for_mysql(): SELECT, UPDATE, REPLACE, CHECK TABLE,
  (almost anything else than INSERT)

* foreign key CASCADE operations

* row_merge_read_clustered_index(): index creation (since MySQL 5.1
  InnoDB Plugin)

* multi-threaded purge (after MySQL 5.5): not sure, but it might fail
  to purge some records

Not all callers of btr_pcur_restore_position() should be affected.
Anything that asserts or checks that restoration succeeds is
unaffected. For example, cursor restoration on the change buffer tree
should always succeed, because access is being protected by additional
latches. Likewise, rollback, or any code accesses data dictionary
tables while holding dict_sys->mutex should be safe.

rb:967 approved by Jimmy Yang
2012-03-08 14:56:22 +02:00
Tor Didriksen
84c6c75979 Merge 5.1 => 5.5
storage/innobase/include/sync0rw.ic:
  Prerequisite for compiling with gcc4 on solaris: ignore result from
  os_compare_and_swap_ulint
storage/myisam/mi_dynrec.c:
  Prerequisite for compiling with gcc4 on solaris: cast to void*
2012-03-06 13:47:57 +01:00
Annamalai Gurusami
621db3d42e The innodb plugin module cannot use DEBUG_SYNC_C facility on Windows.
Taking care of it.
2012-03-01 15:44:23 +05:30
Annamalai Gurusami
7e36c4bdb0 Bug#13635833: MULTIPLE CRASHES IN FOREIGN KEY CODE WITH CONCURRENT DDL/DML
There are two threads.  In one thread, dml operation is going on 
involving cascaded update operation.  In another thread, alter 
table add foreign key constraint is happening.  Under these 
circumstances, it is possible for the dml thread to access a 
dict_foreign_t object that has been freed by the ddl thread.  
The debug sync test case provides the sequence of operations.  
Without fix, the test case will crash the server (because of 
newly added assert).  With fix, the alter table stmt will return 
an error message.  
      
Backporting the fix from MySQL 5.5 to 5.1

rb:961
rb:947
2012-03-01 11:05:51 +05:30
Marko Mäkelä
e74c9b71ab Fix a mistake in the Bug#12861864 fix.
row_drop_table_for_mysql(): Really flag the indexes unavailable before
starting to drop the table.
2012-02-28 21:41:55 +02:00
Marko Mäkelä
1bd0c9b531 Bug#12861864 RACE CONDITION IN BTR_GET_SIZE() AND DROP INDEX/TABLE/DATABASE
also filed as Bug#13146269, Bug#13713178

btr_get_size(): Add mtr_t parameter. Require that the caller S-latches
index->lock. If index->page==FIL_NULL or the index is to be dropped,
return ULINT_UNDEFINED to indicate that the statistics are
unavailable.

dict_update_statistics(): If btr_get_size() returns ULINT_UNDEFINED,
fake the index cardinality statistics.

dict_index_set_page(): Unused function, remove.

row_drop_table_for_mysql(): Before starting to drop the table, mark
the indexes unavailable in the data dictionary cache while holding
index->lock X-latch.

ha_innobase::prepare_drop_index(), ha_innobase::final_drop_index():
When setting index->to_be_dropped, acquire the index->lock X-latch.

rb:960 approved by Jimmy Yang
2012-02-28 14:00:00 +02:00
Marko Mäkelä
2367e79e4c Merge mysql-5.1 to mysql-5.5. 2012-02-28 21:43:08 +02:00
Marko Mäkelä
8235c4e161 Merge mysql-5.1 to mysql-5.5. 2012-02-28 14:04:21 +02:00
Marko Mäkelä
a44974abdc Merge mysql-5.1 to mysql-5.5. 2012-02-27 23:24:36 +02:00
Marko Mäkelä
5682272e1c Remove a bogus BLOB debug assertion that was added in Bug#13721257 fix. 2012-02-27 23:19:14 +02:00
Annamalai Gurusami
c1615df32c Bug#13635833: MULTIPLE CRASHES IN FOREIGN KEY CODE WITH CONCURRENT DDL/DML
There are two threads.  In one thread, dml operation is going on 
involving cascaded update operation.  In another thread, alter 
table add foreign key constraint is happening.  Under these 
circumstances, it is possible for the dml thread to access a 
dict_foreign_t object that has been freed by the ddl thread.  
The debug sync test case provides the sequence of operations.  
Without fix, the test case will crash the server (because of 
newly added assert).  With fix, the alter table stmt will return 
an error message.  
      
rb:947
approved by Jimmy Yang
2012-02-27 17:23:56 +05:30
Jimmy Yang
37d2bf8454 Fix Bug #64432 Port bug fix #54330 from mysql-5.1 to mysql-5.5 2012-02-24 21:21:07 +08:00
Georgi Kodinov
d4c35861e5 merge mysql-5.5->mysql-5.5-security 2012-02-18 10:58:31 +02:00
Georgi Kodinov
9078288a6a merge mysql-5.1->mysql-5.1-security 2012-02-18 10:58:19 +02:00
unknown
abd3c3a9e1 Merge with mysql-5.5 2012-02-17 08:28:28 -06:00
Georgi Kodinov
9f18fc5a06 merged mysql-5.5->mysql-5.5-security 2012-02-17 11:55:36 +02:00
Marko Mäkelä
4ea57c80b2 Merge mysql-5.1 to mysql-5.5. 2012-02-17 11:52:51 +02:00
Georgi Kodinov
387721101c merge mysql-5.1->mysql-5.1-security 2012-02-17 11:52:41 +02:00
Marko Mäkelä
f77329ace9 Bug#13721257 RACE CONDITION IN UPDATES OR INSERTS OF WIDE RECORDS
This bug was originally filed and fixed as Bug#12612184. The original
fix was buggy, and it was patched by Bug#12704861. Also that patch was
buggy (potentially breaking crash recovery), and both fixes were
reverted.

This fix was not ported to the built-in InnoDB of MySQL 5.1, because
the function signatures of many core functions are different from
InnoDB Plugin and later versions. The block allocation routines and
their callers would have to changed so that they handle block
descriptors instead of page frames.

When a record is updated so that its size grows, non-updated columns
can be selected for external (off-page) storage. The bug is that the
initially inserted updated record contains an all-zero BLOB pointer to
the field that was not updated. Only after the BLOB pages have been
allocated and written, the valid pointer can be written to the record.

Between the release of the page latch in mtr_commit(mtr) after
btr_cur_pessimistic_update() and the re-latching of the page in
btr_pcur_restore_position(), other threads can see the invalid BLOB
pointer consisting of 20 zero bytes. Moreover, if the system crashes
at this point, the situation could persist after crash recovery, and
the contents of the non-updated column would be permanently lost.

The problem is amplified by the ROW_FORMAT=DYNAMIC and
ROW_FORMAT=COMPRESSED that were introduced in
innodb_file_format=barracuda in InnoDB Plugin, but the bug does exist
in all InnoDB versions.

The fix is as follows. After a pessimistic B-tree operation that needs
to write out off-page columns, allocate the pages for these columns in
the mini-transaction that performed the B-tree operation (btr_mtr),
but write the pages in a separate mini-transaction (blob_mtr). Do
mtr_commit(blob_mtr) before mtr_commit(btr_mtr). A quirk: Do not reuse
pages that were previously freed in btr_mtr. Only write the off-page
columns to 'fresh' pages.

In this way, crash recovery will see redo log entries for blob_mtr
before any redo log entry for btr_mtr. It will apply the BLOB page
writes to pages that were marked free at that point. If crash recovery
fails to see all of the btr_mtr redo log, there will be some
unreachable BLOB data in free pages, but the B-tree will be in a
consistent state.

btr_page_alloc_low(): Renamed from btr_page_alloc(). Add the parameter
init_mtr. Return an allocated block, or NULL. If init_mtr!=mtr but
the page was already X-latched in mtr, do not initialize the page.

btr_page_alloc(): Wrapper for btr_page_alloc_for_ibuf() and
btr_page_alloc_low().

btr_page_free(): Add a debug assertion that the page was a B-tree page.

btr_lift_page_up(): Return the father block.

btr_compress(), btr_cur_compress_if_useful(): Add the parameter ibool
adjust, for adjusting the cursor position.

btr_cur_pessimistic_update(): Preserve the cursor position when
big_rec will be written and the new flag BTR_KEEP_POS_FLAG is defined.
Remove a duplicate rec_get_offsets() call. Keep the X-latch on
index->lock when big_rec is needed.

btr_store_big_rec_extern_fields(): Replace update_inplace with
an operation code, and local_mtr with btr_mtr. When not doing a
fresh insert and btr_mtr has freed pages, put aside any pages that
were previously X-latched in btr_mtr, and free the pages after
writing out all data. The data must be written to 'fresh' pages,
because btr_mtr will be committed and written to the redo log after
the BLOB writes have been written to the redo log.

btr_blob_op_is_update(): Check if an operation passed to
btr_store_big_rec_extern_fields() is an update or insert-by-update.

fseg_alloc_free_page_low(), fsp_alloc_free_page(),
fseg_alloc_free_extent(), fseg_alloc_free_page_general(): Add the
parameter init_mtr. Return an allocated block, or NULL. If
init_mtr!=mtr but the page was already X-latched in mtr, do not
initialize the page.

xdes_get_descriptor_with_space_hdr(): Assert that the file space
header is being X-latched.

fsp_alloc_from_free_frag(): Refactored from fsp_alloc_free_page().

fsp_page_create(): New function, for allocating, X-latching and
potentially initializing a page. If init_mtr!=mtr but the page was
already X-latched in mtr, do not initialize the page.

fsp_free_page(): Add ut_ad(0) to the error outcomes.

fsp_free_page(), fseg_free_page_low(): Increment mtr->n_freed_pages.

fsp_alloc_seg_inode_page(), fseg_create_general(): Assert that the
page was not previously X-latched in the mini-transaction. A file
segment or inode page should never be allocated in the middle of an
mini-transaction that frees pages, such as btr_cur_pessimistic_delete().

fseg_alloc_free_page_low(): If the hinted page was allocated, skip the
check if the tablespace should be extended. Return NULL instead of
FIL_NULL on failure. Remove the flag frag_page_allocated. Instead,
return directly, because the page would already have been initialized.

fseg_find_free_frag_page_slot() would return ULINT_UNDEFINED on error,
not FIL_NULL. Correct a bogus assertion.

fseg_alloc_free_page(): Redefine as a wrapper macro around
fseg_alloc_free_page_general().

buf_block_buf_fix_inc(): Move the definition from the buf0buf.ic to
buf0buf.h, so that it can be called from other modules.

mtr_t: Add n_freed_pages (number of pages that have been freed).

page_rec_get_nth_const(), page_rec_get_nth(): The inverse function of
page_rec_get_n_recs_before(), get the nth record of the record
list. This is faster than iterating the linked list. Refactored from
page_get_middle_rec().

trx_undo_rec_copy(): Add a debug assertion for the length.

trx_undo_add_page(): Return a block descriptor or NULL instead of a
page number or FIL_NULL.

trx_undo_report_row_operation(): Add debug assertions.

trx_sys_create_doublewrite_buf(): Assert that each page was not
previously X-latched.

page_cur_insert_rec_zip_reorg(): Make use of page_rec_get_nth().

row_ins_clust_index_entry_by_modify(): Pass BTR_KEEP_POS_FLAG, so that
the repositioning of the cursor can be avoided.

row_ins_index_entry_low(): Add DEBUG_SYNC points before and after
writing off-page columns. If inserting by updating a delete-marked
record, do not reposition the cursor or commit the mini-transaction
before writing the off-page columns.

row_build(): Tighten a debug assertion about null BLOB pointers.

row_upd_clust_rec(): Add DEBUG_SYNC points before and after writing
off-page columns. Do not reposition the cursor or commit the
mini-transaction before writing the off-page columns.

rb:939 approved by Jimmy Yang
2012-02-17 11:42:04 +02:00
Marko Mäkelä
2837fba743 Put back a fix that was reverted when reverting the buggy Bug#12612184 fix in
revision-id inaam.rana@oracle.com-20110930110219-vnpaqghj9hm0grds
(revno 3559).
2012-02-17 09:18:53 +02:00
unknown
971fe347dd Bug#64160, Oracle Bug#13698765
The problem was introduced in 5.5.20 by Bug 13116225.  It tried to
protect against downgrading from a version 5.6.4 database that was
created with a page size other than 16k.  Version 5.6.4 supports
page sizes 4k and 8k and stamps that page size into the header page
of each tablespace file.  Version 5.5.20 attempts to read that page
size in the file header.

But it turns out that only the first system tablespace file has a
reliable flags field in the header. So only ibdata1 can be or needs
to be tested for another page size. Extra system tablespace files
like ibdata2, ibdata3, etc do not and should not be tested since the
flags field is unreliable.
2012-02-16 12:13:08 -06:00
Marko Mäkelä
e63d0c916b Fix link error on Windows.
error LNK2001: unresolved external symbol _debug_sync_C_callback_ptr
2012-02-16 15:54:16 +02:00
Kent Boortz
3269f45489 Merge 2012-02-16 12:04:44 +01:00
Kent Boortz
2f755bdf66 Merge 2012-02-16 12:02:53 +01:00
Marko Mäkelä
6ff320e0f6 Merge mysql-5.1 to mysql-5.5. 2012-02-16 12:28:49 +02:00
Marko Mäkelä
4045c9976c Add instrumentation for Bug#13721257 RACE CONDITION IN UPDATES OR INSERTS
OF WIDE RECORDS

row_ins_index_entry_low(), row_upd_clust_rec(): Make a redo log
checkpoint if a DEBUG flag is set. Add DEBUG_SYNC around
btr_store_big_rec_extern_fields().

rb:946 approved by Jimmy Yang
2012-02-16 12:24:11 +02:00
Marko Mäkelä
66b6587206 Correct a few copyright messages. 2012-02-16 12:20:41 +02:00
MySQL Build Team
7a35cb9150 Updated/added copyright headers 2012-02-16 10:48:16 +01:00
Kent Boortz
6a003dd8ef Updated/added copyright headers 2012-02-15 17:21:38 +01:00
Sunny Bains
b566d9a512 BUG#12739098 - 62401: ASSERTION TRX->ERROR_STATE == DB_SUCCESS, QUE0QUE.C LINE 1264 ON TRUNCATE
During FIC error handling the trx->error_state was not being set to DB_SUCCESS
after failure, before attempting the next DDL SQL operation. This reset to
DB_SUCCESS is somewhat of a requirement though not explicitly stated anywhere.
The fix is to reset it to DB_SUCCESS in row0merge.cc if row_merge_rename_indexes
or row_merge_drop_index functions fail, also reset to DB_SUCCESS at trx commit.
				          
rb://935 Approved by Jimmy Yang.
2012-02-10 14:09:12 +11:00
Sunny Bains
ec47cb6034 Merge from mysql-5.1-innodb. 2012-02-10 13:04:10 +11:00
Sunny Bains
297002573c BUG#12739098 - 62401: ASSERTION TRX->ERROR_STATE == DB_SUCCESS, QUE0QUE.C LINE 1264 ON TRUNCATE
During FIC error handling the trx->error_state was not being set to DB_SUCCESS
after failure, before attempting the next DDL SQL operation. This reset to
DB_SUCCESS is somewhat of a requirement though not explicitly stated anywhere.
The fix is to reset it to DB_SUCCESS in row0merge.cc if row_merge_rename_indexes
or row_merge_drop_index functions fail, also reset to DB_SUCCESS at trx commit.

rb://935 Approved by Jimmy Yang.
2012-02-10 12:47:42 +11:00
Georgi Kodinov
91c1c93019 merge mysql-5.5->mysql-5.5-security 2012-02-06 18:26:36 +02:00
Georgi Kodinov
d29ae4918e merged mysql-5.1->mysql-5.1-security 2012-02-06 18:24:51 +02:00
Vasil Dimov
56e3f68c72 Merge mysql-5.1 -> mysql-5.5
The actual Bug#11754376 does not exist in MySQL 5.5 because at startup
we drop entries for temporary tables from InnoDB dictionary cache (only
if ROW_FORMAT is not REDUNDANT). But nevertheless the bug in
normalize_table_name_low() is present so we fix it.
2012-02-06 13:00:41 +02:00
Vasil Dimov
1c4fd3bb54 Fix Bug#11754376 45976: INNODB LOST FILES FOR TEMPORARY TABLES ON
GRACEFUL SHUTDOWN

During startup mysql picks up .frm files from the tmpdir directory and
tries to drop those tables in the storage engine.

The problem is that when tmpdir ends in / then ha_innobase::delete_table()
is passed a string like "/var/tmp//#sql123", then it wrongly normalizes it
to "/#sql123" and calls row_drop_table_for_mysql() which of course fails
to delete the table entry from the InnoDB dictionary cache.
ha_innobase::delete_table() returns an error but nevertheless mysql wipes
away the .frm file and the entry in the InnoDB dictionary cache remains
orphaned with no easy way to remove it.

The "no easy" way to remove it is to create a similar temporary table again,
copy its .frm file to tmpdir under "#sql123.frm" and restart mysqld with
tmpdir=/var/tmp (no trailing slash) - this way mysql will pick the .frm file
after restart and will try to issue drop table for "/var/tmp/#sql123"
(notice do double slash), ha_innobase::delete_table() will normalize it to
"tmp/#sql123" and row_drop_table_for_mysql() will successfully remove the
table entry from the dictionary cache.

The solution is to fix normalize_table_name_low() to normalize things like
"/var/tmp//table" correctly to "tmp/table".

This patch also adds a test function which invokes
normalize_table_name_low() with various inputs to make sure it works
correctly and a mtr test that calls this test function.

Reviewed by:	Marko (http://bur03.no.oracle.com/rb/r/929/)
2012-02-06 12:44:59 +02:00
Ashish Agarwal
b259646460 BUG#11748748: Merge from mysql-5.1 to mysql-5.5 2012-02-03 19:41:03 +05:30
Ashish Agarwal
cfcb5daba8 BUG#11748748 - 37280: CHECK AND REPAIR TABLE REPORT TABLE
CORRUPTED WHEN RUN CONCURRENTLY WITH

ISSUE: Table corruption due to concurrent queries.
       Different threads running check, repair query
       along with insert. Locks not properly acquired
       in repair query. Rows are inserted inbetween
       repair query.

SOLUTION: Mutex lock is acquired before the
          repair call. Concurrent queries wont
          effect the call to repair.
2012-02-03 19:37:00 +05:30
Ashish Agarwal
f0955f9ec7 BUG#11754145 - 45702: IMPOSSIBE TO SPECIFY MYISAM_SORT_BUFFER > 4GB
ON 64 BIT MACHINES

PROBLEM: When sorting index during repair of
         myisam tables, due to improper casting
         of buffer size variables value of myisam_
         sort_buffer_size is not set greater than
         4GB.

SOLUTION: Proper casting of buffer size variable.
          myisam_buffer_size changed to unsigned
          long long to handle size > 4GB on
          linux as well as windows.
2012-02-02 18:17:14 +05:30
Marko Mäkelä
1fbbacfea5 ha_innobase::records_in_range(): Remove a debug assertion
that prohibits an open range (full table).
This assertion catches unnecessary calls to this method, but such calls
are not harming correctness.
2012-02-02 13:54:27 +02:00
Marko Mäkelä
567a422cf9 Merge mysql-5.1 to mysql-5.5. 2012-02-02 13:50:54 +02:00
Marko Mäkelä
e726580b66 Bug#13654923 BOGUS DEBUG ASSERTION IN INDEX CREATION FOR ZERO-LENGTH RECORD
row_merge_buf_write(): Relax the bogus assertion.
2012-02-02 13:38:32 +02:00
Marko Mäkelä
39100cd984 Bug #13651627 Move ut_ad(0) from the beginning to the end of buf_page_print(),
print page dump

buf_page_print(): Remove the ut_ad(0) from the beginning. Add two flags
(enum buf_page_print_flags) that can be bitwise-ORed together:

BUF_PAGE_PRINT_NO_CRASH:
  Do not crash debug builds at the end of buf_page_print().
BUF_PAGE_PRINT_NO_FULL:
  Do not print the full page dump. This can be useful when adding
  diagnostic printout to flushing or to the doublewrite buffer.

trx_sys_doublewrite_init_or_restore_page(): Replace exit(1) with ut_error,
so that we can get a core dump if this extraordinary condition happens.

rb:924 approved by Sunny Bains
2012-02-02 12:31:57 +02:00
Ashish Agarwal
a076427ffd BUG#11756869: Merge from mysql-5.1 to mysql-5.5 2012-02-02 14:25:43 +05:30
Ashish Agarwal
4e096d2a45 BUG#11756869 - 48848: MYISAMCHK DOING SORT RECOVER IN CERTAIN
CASES RESETS DATA POINTER TO SMAL

ISSUE: Myisamchk doing sort recover
       on a table reduces data_file_length.
       Maximum size of data file decreases,
       lesser number of rows are stored.

SOLUTION: Size of data_file_length is
          fixed to the original length.
2012-02-02 14:19:38 +05:30
Ashish Agarwal
52357198d0 BUG#11756869 - 48848: MYISAMCHK DOING SORT RECOVER IN CERTAIN
CASES RESETS DATA POINTER TO SMAL

ISSUE: Myisamchk doing sort recover
       on a table reduces data_file_length.
       Maximum size of data file decreases,
       lesser number of rows are stored.

SOLUTION: Size of data_file_length is
          fixed to the original length.
2012-02-01 11:19:53 +05:30