differences in tables
Certain merge tables were wrongly reported as having incorrect definition:
- Some fields that are 1 byte long (e.g. TINYINT, CHAR(1)), might
be internally casted (in certain cases) to a different type on a
storage engine layer. (affects 4.1 and up)
- If tables in a merge (and a MERGE table itself) had short VARCHAR column (less
than 4 bytes) and at least one (but not all) tables were ALTER'ed (even to an
identical table: ALTER TABLE xxx ENGINE=yyy), table definitions went ouf of
sync. (affects 4.1 only)
This is fixed by relaxing a check for underlying conformance and setting
field type to FIELD_TYPE_STRING in case varchar is shorter than 4
when a table is created.
myisam/mi_create.c:
Added a comment.
mysql-test/r/merge.result:
A test case for bug#26881.
mysql-test/t/merge.test:
A test case for bug#26881.
sql/ha_myisam.cc:
Relaxed some checks performed by check_definition():
As comparing of fulltext keys (and key segments) is not yet implemented,
only return an error in case one of keys is fulltext and other is not.
Otherwise, if both keys are fulltext, accept them as is.
As comparing of spatial keys (and key segments) is not yet implemented,
only return an error in case one of keys is spatial and other is not.
Otherwise, if both keys are spatial, accept them as is.
A workaround to handle situation when field is casted from FIELD_SKIP_ZERO
to FIELD_NORMAL. This could happen only in case field length is 1 and row
format is fixed.
sql/sql_parse.cc:
When a table that has varchar field shorter than 4 is created, field type is
set to FIELD_TYPE_VAR_STRING. Later, when a table is modified using alter
table, field type is changed to FIELD_TYPE_STRING (see Field_string::type).
That means HA_OPTION_PACK_RECORD flag might be lost and thus null_bit might
be shifted by alter table, in other words alter table doesn't create 100%
equal table definition.
This is usually not a problem, since when a table is created/altered,
definition on a storage engine layer is based on one that is passed from
sql layer. But it is a problem for merge engine - null_bit is shifted when
a table (merge or underlying) is altered.
Set field type to FIELD_TYPE_STRING in case FIELD_TYPE_VAR_STRING is shorter
than 4 when a table is created as it is done in Field::type.
incorrect key file for table
In certain cases it could happen that deleting a row could
corrupt an RTREE index.
According to Guttman's algorithm, page underflow is handled
by storing the page in a list for later re-insertion. The
keys from the stored pages have to be inserted into the
remaining pages of the same level of the tree. Hence the
level number is stored in the re-insertion list together
with the page.
In the MySQL RTree implementation the level counts from zero
at the root page, increasing numbers for levels down the tree.
If during re-insertion of the keys the tree height grows, all
level numbers become invalid. The remaining keys will be
inserted at the wrong level.
The fix is to increment the level numbers stored in the
reinsert list after a split of the root block during reinsertion.
myisam/rt_index.c:
Bug#25673 - spatial index corruption, error 126
incorrect key file for table
Added a loop in rtree_delete() to increment the level numbers
stored in the reinsert list after a split of the root block
during reinsertion.
Added comments and DBUG statements.
myisam/rt_key.c:
Bug#25673 - spatial index corruption, error 126
incorrect key file for table
Added DBUG statements.
myisam/rt_split.c:
Bug#25673 - spatial index corruption, error 126
incorrect key file for table
Added DBUG statements.
mysql-test/r/gis-rtree.result:
Bug#25673 - spatial index corruption, error 126
incorrect key file for table
Added the test result.
mysql-test/t/gis-rtree.test:
Bug#25673 - spatial index corruption, error 126
incorrect key file for table
Added a test.
into chilla.local:/home/mydev/mysql-4.1-axmrg
mysql-test/r/symlink.result:
Auto merged
mysql-test/t/symlink.test:
Auto merged
mysql-test/r/myisam.result:
Manual merged
mysql-test/t/myisam.test:
Manual merged
corrupted table
Accessing a table with corrupted column definition results in server
crash.
This is fixed by refusing to open such tables. Affects MyISAM only.
No test case, since it requires crashed table.
myisam/mi_open.c:
Refuse to open MyISAM table with summary columns length bigger than
length of the record.
Accessing fixed record format table with crashed key definition results
in server/myisamchk segmentation fault.
This is fixed by refusing to open such tables. Affects MyISAM only.
No test case, since it requires crashed table.
myisam/mi_open.c:
Refuse to open fixed record format table with key segment that includes
BLOB part (which is true only for tables with crashed key definition).
The function mi_get_pointer_length() computed too small
pointer size for very large tables.
Inserted missing 'else' between the branches for very
large tables.
myisam/mi_create.c:
Bug#24607 - MyISAM pointer size determined incorrectly
Inserted missing 'else' between the branches for very
large tables.
Harmonized literals "(longlong) 1" and "1L" to "ULL(1)"
where they are used for "ulonglong file_length".
mysql-test/r/myisam.result:
Bug#24607 - MyISAM pointer size determined incorrectly
Added the test result.
mysql-test/t/myisam.test:
Bug#24607 - MyISAM pointer size determined incorrectly
Added the test.
Compiler warnings due to non-matching conversion
specifications in format strings in DBUG_PRINT calls,
due to non-used parameters (in non-debug mode), and
due to seemingly uninitialized variables.
Initialized variables, declared parameters unused, and
casted DBUG_PRINT arguments to get rid of warnings.
myisam/mi_range.c:
Bug#25213 - Compiler warnings in MyISAM code
Initialized a variable to get rid of a compiler warning.
myisam/mi_test1.c:
Bug#25213 - Compiler warnings in MyISAM code
Declared an parameter unused to get rid of warnings.
myisam/mi_write.c:
Bug#25213 - Compiler warnings in MyISAM code
Initialized a variable to get rid of a compiler warning.
Casted arguments to DBUG_PRINT to match them with their
format string conversion specification.
myisam/rt_split.c:
Bug#25213 - Compiler warnings in MyISAM code
Initialized variables to get rid of compiler warnings.
Compiler warnings due to non-matching conversion
specifications in format strings in DBUG_PRINT calls.
Fixed DBUG_PRINT format specifiactions.
myisam/mi_packrec.c:
Bug#25208 - Warnings in mi_packrec.c
Fixed DBUG_PRINT format specifiactions.
storage engine returns errno 12
If there is not enough memory to store or update blob record
(while allocating record buffer), myisam marks table as crashed.
With this fix myisam attempts to roll an index back and return
an error, not marking a table as crashed.
Affects myisam tables with blobs only. No test case for this fix.
myisam/mi_dynrec.c:
If there is not enough memory to store or update blob record
(while allocating record buffer), return HA_ERR_OUT_OF_MEM
instead of ENOMEM. In this case storage engine can simply
roll an index back and return an error, not marking table
as crashed.
myisam/mi_update.c:
In some cases do not mark a table as crashed if we run out of
memory. Instead roll an index back and return an error. These
cases are signalled with my_errno set to HA_ERR_OUT_OF_MEM.
myisam/mi_write.c:
In some cases do not mark a table as crashed if we run out of
memory. Instead roll an index back and return an error. These
cases are signalled with my_errno set to HA_ERR_OUT_OF_MEM.
ALTER TABLE DISABLE KEYS doesn't work when modifying the table
ENABLE|DISABLE KEYS combined with another ALTER TABLE option, different
than RENAME TO did nothing. Also, if the table had disabled keys
and was ALTER-ed then the end table was with enabled keys.
Fixed by checking whether the table had disabled keys and enabling them
in the copied table.
myisam/mi_open.c:
Extend mi_indexes_are_disabled to implement return value
2 - Non-unique indexes are disabled
mysql-test/r/alter_table.result:
update result
mysql-test/t/alter_table.test:
update test
sql/sql_table.cc:
When ENABLE|DISABLE index is combined with another option
different than RENAME TO, we should ENABLE|DISABLE the keys of
the modified table. Also when modifying we should preserve the
previous state of the indices.
(This problem exists in 5.0 and 5.1 but since the codebase has
diverged, this fix won't automerge, but the fix will be quite
similar).
A corrupted compressed table could crash the server and
myisamchk.
The data file of an uncompressed table contains just the records.
There is no header in the data file.
However the data file of a compressed table has a header.
The header describes how the table was compressed. This
information is necessary to extract the records from the
compressed data file.
Part of the compressed data file header are the [de]code tables.
They are numeric representations of the Huffman trees used for
coding and decoding. A Huffman tree is a binary tree. Every
node has two childs. A child can be a leaf or a branch. Leaves
contain the decoded value. Branches point to another tree node.
Since the [de]code table is represented as an array of childs,
the branches need to point at a child within the same array.
The corruption of the compressed data file from the bug report
was a couple of branches that pointed outside their array.
This condition had not been correctly checked.
I added some checks for the pointers in the decode tables.
This type of corruption will no longer crash the server or
myisamchk.
No test case. A corrupted compressed table is required.
myisam/mi_packrec.c:
Bug#23139 - myisamchk and mysqld crash when trying to access table
Added some checks for the pointers in the decode tables.
Added comments, DBUG prints, style fixes.
into zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-4.1-maint
configure.in:
Auto merged
mysql-test/t/ps.test:
Auto merged
sql/handler.cc:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/table.cc:
Auto merged
tests/mysql_client_test.c:
Auto merged
myisam/sort.c:
Manual merge.
mysql-test/r/innodb_mysql.result:
Manual merge.
mysql-test/t/innodb_mysql.test:
Manual merge.
mysys/mf_iocache.c:
Manual merge.
When compiling with a default key block size greater than the
smallest key block size used in a table, checking that table
failed with bogus errors. The table was marked corrupt. This
affected myisamchk and the server.
The problem was that the default key block size was used at
some places where sizes less or equal to the block size of the
index in check was required.
We do now use the key block size of the particular index
when checking.
A test case is available for later versions only.
myisam/mi_check.c:
Bug#22119 - Changing MI_KEY_BLOCK_LENGTH makes a wrong myisamchk
Changed check_k_link() and chk_index_down() to use the block
size of the index in check or MI_MIN_KEY_BLOCK_LENGTH where
required. Formerly myisam_block_size or MYISAM_SHARE::blocksize
was used wrongly.
really damaged MyISAM tables
When unpacking a blob column from broken row server crash
could happen. This could rather happen when trying to repair
a table using either REPAIR TABLE or myisamchk, though it
also could happend when trying to access broken row using
other SQL statements like SELECT if table is not marked as
crashed.
Fixed ulong overflow when trying to extract blob from
broken row.
Affects MyISAM only.
myisam/mi_dynrec.c:
Fixed ulong overflow when trying to extract blob from
broken row. It happens when there are not enough bytes
to store blob length in `from' buffer. In this case
(ulong) (from_end - from) - size_length value is huge,
close to ULONG_MAX.
Repair table could crash a server if there is not sufficient
memory (myisam_sort_buffer_size) to operate. Affects not only
repair, but also all statements that use create index by sort:
repair by sort, parallel repair, bulk insert.
Return an error if there is not sufficient memory to store at
least one key per BUFFPEK.
Also fixed memory leak if thr_find_all_keys returns an error.
myisam/sort.c:
maxbuffer is number of BUFFPEK-s for repair. It is calculated
as records / keys. keys is number of keys that can be stored
in memory (myisam_sort_buffer_size). There must be sufficient
memory to store both BUFFPEK-s and keys. It was checked
correctly before this patch. However there is another
requirement that wasn't checked: there must be sufficient
memory for at least one key per BUFFPEK, otherwise repair
by sort/parallel repair cannot operate.
Return an error if there is not sufficient memory to store at
least one key per BUFFPEK.
Also fixed memory leak if thr_find_all_keys returns an error.
mysql-test/r/repair.result:
A test case for BUG#23175.
mysql-test/t/repair.test:
A test case for BUG#23175.
set. This has always worked because when flag is !=0 then
HA_VAR_LENGTH_KEY is always set. Therefore, a test case cannot
reveal a faulty behavior.
Fix for bug#23074: typo in myisam/sort.c
myisam/sort.c:
fix typo. Nevertheless, it has worked as expected
because when a bit in flag is set HA_VAR_LENGTH_KEY has
been always set too. Actually, no problem exposed through
DDL.
into chilla.local:/home/mydev/mysql-4.1-bug8283-one
myisam/mi_check.c:
Auto merged
myisam/mi_packrec.c:
Auto merged
myisam/sort.c:
Auto merged
mysql-test/r/myisam.result:
Bug#8283 - OPTIMIZE TABLE causes data loss
Manual merge
mysql-test/t/myisam.test:
Bug#8283 - OPTIMIZE TABLE causes data loss
Manual merge
OPTIMIZE TABLE with myisam_repair_threads > 1 performs a non-quick
parallel repair. This means that it does not only rebuild all
indexes, but also the data file.
Non-quick parallel repair works so that there is one thread per
index. The first of the threads rebuilds also the new data file.
The problem was that all threads shared the read io cache on the
old data file. If there were holes (deleted records) in the table,
the first thread skipped them, writing only contiguous, non-deleted
records to the new data file. Then it built the new index so that
its entries pointed to the correct record positions. But the other
threads didn't know the new record positions, but put the positions
from the old data file into the index.
The new design is so that there is a shared io cache which is filled
by the first thread (the data file writer) with the new contiguous
records and read by the other threads. Now they know the new record
positions.
Another problem was that for the parallel repair of compressed
tables a common bit_buff and rec_buff was used. I changed it so
that thread specific buffers are used for parallel repair.
A similar problem existed for checksum calculation. I made this
multi-thread safe too.
include/my_sys.h:
Bug#8283 - OPTIMIZE TABLE causes data loss
Redesign of io_cache_share.
include/myisam.h:
Bug#8283 - OPTIMIZE TABLE causes data loss
Redesign of checksum calculation in mi_check.c.
'calc_checksum' is now in myisamdef.h:st_mi_sort_param.
myisam/mi_check.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Implemented a new parallel repair design.
Using a synchronized shared read/write cache.
Allowed for thread specific bit_buff, rec_buff, and calc_checksum.
myisam/mi_open.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added DBUG output.
myisam/mi_packrec.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Allowed for thread specific bit_buff and rec_buff.
myisam/myisamdef.h:
Bug#8283 - OPTIMIZE TABLE causes data loss
Commented on checksum calculation variables.
Allowed for thread specific bit_buff.
Added DBUG output for better table crash detection.
myisam/sort.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added implications of the new parallel repair design.
Renamed 'info' -> 'sort_param'.
Added DBUG output.
mysql-test/r/myisam.result:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added test results.
mysql-test/t/myisam.test:
Bug#8283 - OPTIMIZE TABLE causes data loss
Added test cases.
mysys/mf_iocache.c:
Bug#8283 - OPTIMIZE TABLE causes data loss
Redesign of io_cache_share.
We do now allow a writer to synchronize himself with the
readers of a shared cache. When all threads join in the lock,
the writer copies the data from his write buffer to the shared
read buffer.
Fix: using charset-aware functions cs->cset->lengthsp() and cs->cset->fill()
instead of single byte code which is not UCS2 compatible.
myisam/mi_key.c:
Using character-set aware functions to trim and append spaces.
myisam/mi_open.c:
Initialize charset for BINARY/VARBINARY to my_charset_bin,
instead of NULL
mysql-test/r/ctype_ucs.result:
Adding test case
mysql-test/t/ctype_ucs.test:
Adding test case
into mysql.com:/home/svoj/devel/mysql/merge/mysql-4.1-engines
mysql-test/r/myisam.result:
Auto merged
mysql-test/t/myisam.test:
Auto merged
sql/table.cc:
Auto merged
RTree keys are really different from BTree and need specific
paramters to be set by optimizer to work.
Sometimes optimizer doesn't set those properly.
Here we decided just to add code to check that the parameters
are correct. Hope to fix optimizer sometimes.
myisam/mi_range.c:
return the error if min_key is NULL
mysql-test/r/gis-rtree.result:
test result
mysql-test/t/gis-rtree.test:
test case
Deletes on a big index could crash the index when it needs to
shrink.
Put a forgotten negation operator in.
No test case. It is too big for the test suite. And it does not
work with 4.0, only with higher versions. It is attached to the
bug report.
myisam/mi_delete.c:
Bug#22384 - DELETE FROM table causes "Incorrect key file for table"
Put a negation operator ('!') before _mi_get_last_key() in del().
It returns NULL on error, non-NULL on success.
