into chilla.local:/home/mydev/mysql-4.1-bug8283-one
myisam/mi_check.c:
Auto merged
myisam/mi_packrec.c:
Auto merged
myisam/sort.c:
Auto merged
mysql-test/r/myisam.result:
Bug#8283 - OPTIMIZE TABLE causes data loss
Manual merge
mysql-test/t/myisam.test:
Bug#8283 - OPTIMIZE TABLE causes data loss
Manual merge
OPTIMIZE TABLE with myisam_repair_threads > 1 performs a non-quick
parallel repair. This means that it does not only rebuild all
indexes, but also the data file.
Non-quick parallel repair works so that there is one thread per
index. The first of the threads rebuilds also the new data file.
The problem was that all threads shared the read io cache on the
old data file. If there were holes (deleted records) in the table,
the first thread skipped them, writing only contiguous, non-deleted
records to the new data file. Then it built the new index so that
its entries pointed to the correct record positions. But the other
threads didn't know the new record positions, but put the positions
from the old data file into the index.
The new design is so that there is a shared io cache which is filled
by the first thread (the data file writer) with the new contiguous
records and read by the other threads. Now they know the new record
positions.
Another problem was that for the parallel repair of compressed
tables a common bit_buff and rec_buff was used. I changed it so
that thread specific buffers are used for parallel repair.
A similar problem existed for checksum calculation. I made this
multi-thread safe too.
include/my_sys.h:
Bug#8283 - OPTIMIZE TABLE causes data loss
Redesign of io_cache_share.
include/myisam.h:
Bug#8283 - OPTIMIZE TABLE causes data loss
Redesign of checksum calculation in mi_check.c.
'calc_checksum' is now in myisamdef.h:st_mi_sort_param.
myisam/mi_check.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Implemented a new parallel repair design.
Using a synchronized shared read/write cache.
Allowed for thread specific bit_buff, rec_buff, and calc_checksum.
myisam/mi_open.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added DBUG output.
myisam/mi_packrec.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Allowed for thread specific bit_buff and rec_buff.
myisam/myisamdef.h:
Bug#8283 - OPTIMIZE TABLE causes data loss
Commented on checksum calculation variables.
Allowed for thread specific bit_buff.
Added DBUG output for better table crash detection.
myisam/sort.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added implications of the new parallel repair design.
Renamed 'info' -> 'sort_param'.
Added DBUG output.
mysql-test/r/myisam.result:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added test results.
mysql-test/t/myisam.test:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added test cases.
mysys/mf_iocache.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Redesign of io_cache_share.
We do now allow a writer to synchronize himself with the
readers of a shared cache. When all threads join in the lock,
the writer copies the data from his write buffer to the shared
read buffer.
into mysql.com:/home/svoj/devel/mysql/merge/mysql-4.1-engines
mysql-test/r/myisam.result:
Auto merged
mysql-test/t/myisam.test:
Auto merged
sql/table.cc:
Auto merged
Deletes on a big index could crash the index when it needs to
shrink.
Put a forgotten negation operator in.
No test case. It is too big for the test suite. And it does not
work with 4.0, only with higher versions. It is attached to the
bug report.
myisam/mi_delete.c:
Bug#22384 - DELETE FROM table causes "Incorrect key file for table"
Put a negation operator ('!') before _mi_get_last_key() in del().
It returns NULL on error, non-NULL on success.
into chilla.local:/home/mydev/mysql-4.0-bug14400
mysql-test/r/myisam.result:
Auto merged
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Manual merge
mysql-test/t/myisam.test:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Manual merge
into chilla.local:/home/mydev/mysql-4.0-bug14400
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Manual merge
into chilla.local:/home/mydev/mysql-4.1-bug14400-monty
BitKeeper/etc/ignore:
auto-union
include/my_global.h:
Auto merged
myisam/mi_rkey.c:
Manual null merge as a better fix is already present.
mysql-test/r/myisam.result:
Manual null merge as a better fix is already present.
mysql-test/t/myisam.test:
Manual null merge as a better fix is already present.
sql/sql_select.cc:
Manual merge of purify improvements.
into chilla.local:/home/mydev/mysql-4.1-bug14400
mysql-test/r/myisam.result:
Auto merged
mysql-test/t/myisam.test:
Auto merged
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Manual merge
into chilla.local:/home/mydev/mysql-4.1-bug14400
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Manual merge from 4.0
mysql-test/r/myisam.result:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Manual merge from 4.0
mysql-test/t/myisam.test:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Manual merge from 4.0
"concurrent insert"
Additional fix for full keys and test case.
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Additional fix for full keys.
mysql-test/r/myisam.result:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Additional results.
mysql-test/t/myisam.test:
Bug#14400 - Query joins wrong rows from table which is subject of
"concurrent insert"
Additional test case.
The problem is that on some Mac OS X-es the file writing/reading
call with zero bytes to read/write returns error.
So here i try to eliminate that kinds of calls.
myisam/mi_check.c:
zero length copying avoided
mysys/my_chsize.c:
no file operations if it's not necessary
into chilla.local:/home/mydev/mysql-4.1-bug14400
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is
subject of "concurrent insert"
Manual merge from 4.0.
mysql-test/r/myisam.result:
Bug#14400 - Query joins wrong rows from table which is
subject of "concurrent insert"
Manual merge from 4.0.
mysql-test/t/myisam.test:
Bug#14400 - Query joins wrong rows from table which is
subject of "concurrent insert"
Manual merge from 4.0.
subject of "concurrent insert"
Better fix by Monty: "The previous bug fix didn't work
when using partial keys."
mysql-test/r/myisam.result:
Bug#14400 - Query joins wrong rows from table which is
subject of "concurrent insert"
Added test result
mysql-test/t/myisam.test:
Bug#14400 - Query joins wrong rows from table which is
subject of "concurrent insert"
Added test case
table results in table corrupt
Fulltext key has always two keysegs, thus we need to update
FT_SEGS (last) element from seg array in case of compressed table.
Also we must update ft2_keyinfo.
myisam/mi_packrec.c:
Fulltext key has always two keysegs, thus we need to update
FT_SEGS (last) element from seg array in case of compressed table.
Also we must update ft2_keyinfo.
Fixed by moving update_key_parts() down to be after write_index().
myisam/sort.c:
write_index() collects index statistic which is further used in
update_key_parts(). Thus update_key_parts() must be called after
write_index().
mysql-test/r/repair.result:
Test case for bug#18874.
mysql-test/t/repair.test:
Test case for bug#18874.
The previous bug fix didn't work when using partial keys.
Don't use GNUC min/max operations are they are depricated.
Fixed valgrind warning
BitKeeper/etc/ignore:
Added */.libs/*
include/my_global.h:
Don't use GNUC min/max operations are they are depricated
myisam/mi_rkey.c:
Better bug fix for #14400 "Query joins wrong rows from table which is subject of "concurrent insert""
The previous bug fix didn't work when using partial keys.
myisam/mi_test_all.res:
Updated results to match mi_test_all.sh
myisam/mi_test_all.sh:
Removed confusing warning
mysql-test/r/myisam.result:
Added test case for #14400
mysql-test/t/myisam.test:
Added test case for #14400
sql/sql_select.cc:
Fixed valgrind warning (in field_string::val_int())
myisam/mi_uniue.c:mi_check_unique() should skip trailing spaces comparing
TEXT and VARTTEXT key segments.
myisam/mi_unique.c:
Fix for bug #20709: Collation not used in group by on 4.1.
myisam/mi_uniue.c:mi_check_unique() should skip trailing spaces comparing
TEXT and VARTTEXT key segments.
Example: assume, we have a 'char(200) collate utf8_unicode_ci' field,
there are two records with _utf8"0x65" and _utf8"0xC3A9" characters;
these values are equal according
to the utf8_unicode_ci collation, but two 600 byte length corresponding keys:
"0x65<0x20 repeats 599 times>" and "0xC3A9<0x20 repeats 598 times>" are not
equal if we count trailing spaces and it may cause inconsequent behavior.
So, let's pass 1 as the skip_end_space parameter value to the mi_compare_text()
function for proper TEXT and VARTTEXT key segments comparison.
mysql-test/r/ctype_utf8.result:
Fix for bug #20709: Collation not used in group by on 4.1.
- test results.
mysql-test/t/ctype_utf8.test:
Fix for bug #20709: Collation not used in group by on 4.1.
- test case.
Problem described in this bug report affects MyISAM tables only.
Running mysqld --flush instructs mysqld to sync all changes to disk
after each SQL statement. It worked well for INSERT and DELETE
statements, but it did sync for UPDATE only in case if there was
index change (change of colum that has an index). If no updated column
has an index, data wasn't synced to disk.
This fix makes UPDATE statement to sync data to disk even if there is
no index change (that is only data change) and mysqld is run with
--flush option.
myisam/mi_update.c:
Every myisam function that updates myisam table must end with
call to _mi_writeinfo(). If operation (second param of
_mi_writeinfo()) is not 0 it sets share->changed to 1, that is
flags that data has changed. If operation is 0, this function
equals to no-op in this case.
mi_update() must always pass !0 value as operation, since even if
there is no index change there could be data change.
Fixed a possible problem with reading of dynamic records
when a write cache is active. The cache must be flushed
whenever a part of the file in the write cache is to be
read.
Added a read optimization to _mi_read_dynamic_record().
No test case. This was a hypothetical but existing problem.
myisam/mi_dynrec.c:
Bug#20719 - Reading dynamic records with write buffer could fail
Fixed a possible problem with reading of dynamic records
when a write cache is active. The cache must be flushed
whenever a part of the file in the write cache is to be
read. This must be done before the read of the header
and before the read of the rest block.
Renamed the 'flag' and 'skipp_deleted_blocks' variables.
Added a read optimization to _mi_read_dynamic_record()
that was present in _mi_read_rnd_dynamic_record() already.
After _mi_get_block_info() we have some bytes of the record
in the header buffer already. No need to read them again.
"temporary table with data directory option fails"
myisam should not use user-specified table name when creating
temporary tables and use generated connection specific real name.
Test included.
myisam/mi_create.c:
Bug#8706
When creating a temporary table with directory override, ensure that
the real filename is using the hidden temporary name otherwise
multiple clients cannot have same named temporary tables without
conflict.
mysql-test/r/myisam.result:
Bug#8706
Test for bug
mysql-test/t/myisam.test:
Bug#8706
Test for bug
into mysql.com:/home/mydev/mysql-4.1-bug14400
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is subject of "concurrent insert"
Manual merge
sql/sql_class.cc:
Bug#14400 - Query joins wrong rows from table which is subject of "concurrent insert"
Manual merge
It was possible that fetching a record by an exact key value
(including the record pointer) could return a record with a
different key value. This happened only if a concurrent insert
added a record with the searched key value after the fetching
statement locked the table for read.
The search succeded on the key value, but the record was
rejected as it was past the file length that was remembered
at start of the fetching statement. With other words it was
rejected as being a concurrently inserted record.
The action to recover from this problem was to fetch the
record that is pointed at by the next key of the index.
This was repeated until a record below the file length was
found.
I do now avoid this loop if an exact match was searched.
If this match is beyond the file length, it is now treated
as "key not found". There cannot be another key with the
same record pointer.
myisam/mi_rkey.c:
Bug#14400 - Query joins wrong rows from table which is subject of "concurrent insert"
Added a check for exact key match before searching for
the next key that was not concurrently inserted. If an
exact key match finds a concurrently inserted row, this
must be treated as "key not found".
sql/sql_class.cc:
Bug#14400 - Query joins wrong rows from table which is subject of "concurrent insert"
Fixed some DBUG_ENTER strings.
A corrupt table with dynamic record format can crash the
server when trying to select from it.
I fixed the crash that resulted from the particular type
of corruption that has been reported for this bug.
No test case. To test it, one needs a table with a very special
corruption. The bug report contains a file with such a table.
myisam/mi_dynrec.c:
Bug#19835 - Binary copy of corrupted tables crash the server when issuing a query
Added a protection against corrupted records. A dynamic
record header with invalid 'next' pointer could trigger
the assert in _mi_get_block_info(). Now I avoid this by
reporting a corruption error.
CHECK TABLE could complain about a fully intact spatial index.
A wrong comparison operator was used for table checking.
The result was that it checked for non-matching spatial keys.
This succeeded if at least two different keys were present,
but failed if only the matching key was present.
I fixed the key comparison.
myisam/mi_check.c:
Bug#17877 - Corrupted spatial index
Fixed the comparison operator for checking a spatial index.
Using MBR_EQUAL | MBR_DATA to compare for equality and
include the data pointer in the comparison. The latter
finds the index entry that points to the current record.
This is necessary for non-unique indexes.
The old operator, SEARCH_SAME, is unknown to the rtree
search functions and handled like MBR_DISJOINT.
myisam/mi_key.c:
Bug#17877 - Corrupted spatial index
Added a missing DBUG_RETURN.
myisam/rt_index.c:
Bug#17877 - Corrupted spatial index
Included the data pointer in the copy of the search key.
This is necessary for searching the index entry that points
to a specific record if the search_flag contains MBR_DATA.
myisam/rt_mbr.c:
Bug#17877 - Corrupted spatial index
Extended the RT_CMP() macro with an assert for an
unexpected comparison operator.
mysql-test/r/gis-rtree.result:
Bug#17877 - Corrupted spatial index
The test result.
mysql-test/t/gis-rtree.test:
Bug#17877 - Corrupted spatial index
The test case.
Very complex select statements can create temporary tables
that are too big to be represented as a MyISAM table.
This was not checked at table creation time, but only at
open time. The result was an attempt to delete the
"impossible" table.
But if the server is built --with-raid, MyISAM tries to
open the table before deleting the files. It needs to find
out if the table uses the raid support and how many raid
chunks there are. This is done with an open "for repair",
which will almost always succeed.
But in this case we have an "impossible" table. The open
failed. Hence the files were not deleted. Also the error
message was a bit unspecific.
I turned an open error in this situation into the assumption
of having no raid support on the table. Thus the normal data
file is tried to be deleted. This may however leave existing
raid chunks behind.
I also added a check in mi_create() to prevent the creation
of an "impossible" table. A more decriptive error message is
given in this case.
No test case. The required select statement is way too
large for the test suite. I added a test script to the
bug report.
myisam/mi_create.c:
Bug#11824 - internal /tmp/*.{MYD,MYI} files remain, causing subsequent queries to fail
Added a check to mi_create() that the table description
header of the index file does not exceed 64KB. The header
has only 16 bits to encode its length.
myisam/mi_delete_table.c:
Bug#11824 - internal /tmp/*.{MYD,MYI} files remain, causing subsequent queries to fail
Interpret error in table open as not having a raid
configuration on the tbale. Thus try to delete the
normal data file, but leave behind raid chunks if
they exist.
myisam_ftdump options: --help first, then rest in
lexical order.
myisam/myisam_ftdump.c:
myisam_ftdump options: --help first, then rest in
lexical order.
Retrieving data from compressed MyISAM table which is bigger than 4G on 32-bit box
with mmap() support results in server crash.
mmap() accepts length of bytes to be mapped in second param, which is 32-bit
size_t. But we pass data_file_length, which is 64-bit my_off_t. As a result only
first data_file_length % 4G were mapped.
This fix adds additional condition for mmap() usage, that is use mmap() for
compressed table which size is no more than 4G on 32-bit platform.
myisam/mi_packrec.c:
Use mmap() for compressed table which size is no more than 4G on 32-bit platform.
Whenever 'myisamchk' needed to recreate a table,
the auto increment information was lost.
Now the forgotten element of the table creation
information is set correctly.
myisam/mi_check.c:
Bug#10405 - myisamchk damages auto_increment columns when changing character set
Added some DBUG statements.
Enabled 'with_auto_increment' in 'create_info'
when recreating a table.
Bug #17705 "FT Index corruption occurs with UTF8 data..."
(Actually, the bug had nothing to do with FT index but with general key compression)
myisam/mi_search.c:
Fix error in prefix compression of keys in MyISAM when key length changed from 254 -> 255
mysql-test/r/ctype_utf8.result:
Test of fix for key compression bug
mysql-test/t/ctype_utf8.test:
Test of fix for key compression bug
For "count(*) while index_column = value" an index read
is done. It consists of an index scan and retrieval of
each key.
For efficiency reasons the index scan stores the key in
the special buffer 'lastkey2' once only. At the first
iteration it notes this fact with the flag
HA_STATE_RNEXT_SAME in 'info->update'.
For efficiency reasons, the key retrieval for blobs
does not allocate a new buffer, but uses 'lastkey2'...
Now I clear the HA_STATE_RNEXT_SAME flag whenever the
buffer has been polluted. In this case, the index scan
copies the key value again (and sets the flag again).
include/my_base.h:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Changed the comment for HA_STATE_RNEXT_SAME as a warning
for future uses.
myisam/mi_delete.c:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Removing the flag HA_STATE_RNEXT_SAME from info->update
if info->lastkey2 was reused for another purpose than
index scanning.
myisam/mi_key.c:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Removing the flag HA_STATE_RNEXT_SAME from info->update
if info->lastkey2 was reused for another purpose than
index scanning.
myisam/mi_rnext_same.c:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Removed trailing space and fixed a comment.
myisam/mi_unique.c:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Removing the flag HA_STATE_RNEXT_SAME from info->update
if info->lastkey2 was reused for another purpose than
index scanning.
myisam/mi_update.c:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Removing the flag HA_STATE_RNEXT_SAME from info->update
if info->lastkey2 was reused for another purpose than
index scanning.
myisam/mi_write.c:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Removing the flag HA_STATE_RNEXT_SAME from info->update
if info->lastkey2 was reused for another purpose than
index scanning.
mysql-test/r/myisam.result:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Added test result.
mysql-test/t/myisam.test:
Bug#14980 - COUNT(*) incorrect on MyISAM table with certain INDEX
Added test.
