empty result when using DESC
Problem: fetching MyISAM keys we copy a key block pointer to the end of the key buffer.
However, we don't take into account the pointer length calculatig the buffer size,
that may leads to memory overwriting and in turn to unpredictable results.
Fix: increase key buffer size by length of the key block pointer.
Note: no simple test case.
myisam/mi_open.c:
Fix for bug #33758: Got query result when using ORDER BY ASC, but
empty result when using DESC
- increase possible maximum key length by size of the key block pointer,
as it's copied into the key buffer in the get_key() MyISAM functions.
at page 1024 with ucs2_bin
Inserting strings with a common prefix into a table with
characterset UCS2 corrupted the table.
An efficient search method was used, which compares end space
with ASCII blank. This doesn't work for character sets like UCS2,
which do not encode blank like ASCII does.
Use the less efficient search method _mi_seq_search()
for charsets with mbminlen > 1.
myisam/mi_open.c:
Bug#32705 - myisam corruption: Key in wrong position
at page 1024 with ucs2_bin
Use _mi_seq_search() for charsets with mbminlen > 1.
mysql-test/r/myisam.result:
Bug#32705 - myisam corruption: Key in wrong position
at page 1024 with ucs2_bin
Added test result.
mysql-test/t/myisam.test:
Bug#32705 - myisam corruption: Key in wrong position
at page 1024 with ucs2_bin
Added test.
into mysql.com:/home/gluh/MySQL/Merge/5.0-opt
client/mysql.cc:
Auto merged
client/mysqltest.c:
Auto merged
include/mysql_com.h:
Auto merged
libmysql/CMakeLists.txt:
Auto merged
myisam/mi_check.c:
Auto merged
mysql-test/r/delayed.result:
Auto merged
mysql-test/r/innodb.result:
Auto merged
mysql-test/r/myisam.result:
Auto merged
mysql-test/r/ps.result:
Auto merged
mysql-test/t/merge.test:
Auto merged
mysql-test/t/myisam.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/type_datetime.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/field.cc:
Auto merged
sql/ha_myisam.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/set_var.h:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
mysql-test/r/func_misc.result:
manual merge
mysql-test/r/innodb_mysql.result:
manual merge
mysql-test/t/func_misc.test:
manual merge
mysql-test/t/innodb_mysql.test:
manual merge
sql/sql_insert.cc:
manual merge
into ramayana.hindu.god:/home/tsmith/m/bk/maint/50
client/mysqldump.c:
Auto merged
include/my_sys.h:
Auto merged
libmysql/CMakeLists.txt:
Auto merged
libmysql/Makefile.shared:
Auto merged
myisam/ft_boolean_search.c:
Auto merged
myisam/sort.c:
Auto merged
mysql-test/t/cast.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/field.cc:
Auto merged
sql/ha_myisam.cc:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/set_var.h:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/mysql-test-run.pl:
Manual merge
mysql-test/r/ctype_ucs.result:
Manual merge
mysql-test/r/func_misc.result:
Manual merge
mysql-test/t/binlog_killed.test:
Manual merge
mysql-test/t/ctype_ucs.test:
Manual merge
mysql-test/t/func_misc.test:
Manual merge
sql/item_strfunc.h:
Manual merge
strings/ctype-simple.c:
Manual merge
SPATIAL key is fine actually, but the chk_key() function
mistakenly returns error. It tries to compare checksums
of btree and SPATIAL keys while the checksum for the SPATIAL isn't
calculated (always 0). Same thing with FULLTEXT keys is handled
using full_text_keys counter, so fixed by counting both
SPATIAL and FULLTEXT keys in that counter.
myisam/mi_check.c:
Bug #30284 spatial key corruption
full_text_keys counts both FULL_TEXT and SPATIAL keys
mysql-test/r/gis.result:
Bug #30284 spatial key corruption
test result
mysql-test/t/gis.test:
Bug #30284 spatial key corruption.
test case
into mysql.com:/home/gluh/MySQL/Merge/5.0-opt
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
mysql-test/r/select.result:
manual merge
mysql-test/t/select.test:
manual merge
Post-pushbuild fix
Added a purecov comment and a test for coverage of parallel
enable keys.
myisam/mi_check.c:
Bug#4692 - DISABLE/ENABLE KEYS waste a space
Added purecov comment.
mysql-test/r/myisam.result:
Bug#4692 - DISABLE/ENABLE KEYS waste a space
Added test result.
mysql-test/t/myisam.test:
Bug#4692 - DISABLE/ENABLE KEYS waste a space
Added test for coverage of parallel enable keys.
into mysql.com:/home/svoj/devel/mysql/BUG31277/mysql-5.0-engines
myisam/mi_check.c:
Auto merged
myisam/mi_open.c:
Auto merged
myisam/mi_packrec.c:
Auto merged
myisam/myisamchk.c:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
sql/mysql_priv.h:
Auto merged
include/mysql_com.h:
Use local.
sql-common/client.c:
Use local.
When we insert a record into MYISAM table which is almost 'full',
we first write record data in the free space inside a file, and then
check if we have enough space after the end of the file.
So if we don't have the space, table will left corrupted.
Similar error also happens when we updata MYISAM tables.
Fixed by modifying write_dynamic_record and update_dynamic_record functions
to check for free space before writing parts of a record
BitKeeper/etc/ignore:
Added libmysql_r/client_settings.h libmysqld/ha_blackhole.cc to the ignore list
myisam/mi_dynrec.c:
Bug #31305 myisam tables crash when they are near capacity.
now we check space left in table in write_dynamic_record
and update_dynamic_record functions.
If we don't have enough room for the new (updated) record, return with the
error.
mysql-test/r/almost_full.result:
New BitKeeper file ``mysql-test/r/almost_full.result''
mysql-test/t/almost_full.test:
New BitKeeper file ``mysql-test/t/almost_full.test''
With certain data sets (when compressed record length gets bigger than
uncompressed) myisamchk --unpack may corrupt data file.
Fixed that record length was wrongly restored from compressed table.
myisam/mi_check.c:
With compressed tables compressed record length may be bigger than
pack_reclength, thus we may allocate insufficient memory for record
buffer.
Let single function allocate record buffer, performing needed record
length calculations.
Still, it is not doable with parallel repair, as it allocates needed
record buffers at once. For parellel repair added better record length
calculation.
myisam/mi_open.c:
When calculating record buffer size, take into account that compressed
record length may be bigger than uncompressed.
myisam/mi_packrec.c:
With certain data set share->max_pack_length (compressed record length)
may be bigger than share->base.pack_reclength (packed record length).
set_if_bigger(pack_reclength, max_pack_length) in this case causes
myisamchk --unpack to write extra garbage, whereas pack_reclength
remains the same in new index file. As a result we get unreadable
table.
myisam/myisamchk.c:
With compressed tables compressed record length may be bigger than
pack_reclength, thus we may allocate insufficient memory for record
buffer.
Let single function allocate record buffer, performing needed record
length calculations.
mysql-test/mysql-test-run.pl:
Environment variables to execute myisamchk and myisampack.
mysql-test/r/myisampack.result:
New BitKeeper file ``mysql-test/r/myisampack.result''
mysql-test/t/myisampack.test:
New BitKeeper file ``mysql-test/t/myisampack.test''
Disabling and enabling indexes on a non-empty table grows the
index file.
Disabling indexes just sets a flag per non-unique index and does not
free the index blocks of the affected indexes. Re-enabling indexes
creates new indexes with new blocks. The old blocks remain unused
in the index file.
Fixed by dropping and re-creating all indexes if non-empty disabled
indexes exist when enabling indexes. Dropping all indexes resets
the internal end-of-file marker to the end of the index file header.
It also clears the root block pointers of every index and clears the
deleted blocks chains. This way all blocks are declared as free.
myisam/mi_check.c:
Bug#4692 - DISABLE/ENABLE KEYS waste a space
Added function mi_drop_all_indexes() to support drop of all indexes
in case we want to re-enable non-empty disabled indexes.
Changed mi_repair(), mi_repair_by_sort(), and mi_repair_parallel()
to use the new function instead of duplicate drop index code.
mysql-test/r/myisam.result:
Bug#4692 - DISABLE/ENABLE KEYS waste a space
Added test result.
mysql-test/t/myisam.test:
Bug#4692 - DISABLE/ENABLE KEYS waste a space
Added test.
fulltext index
Having a table with broken multibyte characters may cause fulltext
parser dead-loop.
Since normally it is not possible to insert broken multibyte sequence
into a table, this problem may arise only if table is damaged.
Affected statements are:
- CHECK/REPAIR against damaged table with fulltext index;
- boolean mode phrase search against damaged table with or
without fulltext inex;
- boolean mode searches without index;
- nlq searches.
No test case for this fix. Affects 5.0 only.
myisam/ft_parser.c:
When skipping leading spaces, skip broken characters as well (broken
characters a identified by mbl == 0).
Fulltext boolean mode phrase search may crash server on platforms
where size of pointer is not equal to size of unsigned integer
(in other words some 64-bit platforms).
The problem was integer overflow.
Affects 4.1 only.
myisam/ft_boolean_search.c:
my_match_t::beg is unsigned int, that means type of expression
(m[0].beg - 1) has unsigned type too. It may happen that instr()
finds substring in the beggining of passed string, returning
m[0].beg equal to 0. In this case value of expression (m[0].beg - 1)
is equal to MAX_UINT.
This is not a problem on platforms where sizeof(pointer) equals to
sizeof(uint). That means ptr[(uint)-1] = ptr[(uint)MAX_UINT] = ptr - 1.
On some 64-bit platforms where sizeof(pointer) is 8 and sizeof(uint)
is 4, wrong address gets accessed. In other words ptr[(uint)-1] is
equal to ptr + MAX_UINT.
mysql-test/r/fulltext.result:
A test case for BUG#11392.
mysql-test/t/fulltext.test:
A test case for BUG#11392.
into mysql.com:/home/gluh/MySQL/Merge/5.0-opt
client/mysqldump.c:
Auto merged
include/config-win.h:
Auto merged
libmysql/libmysql.c:
Auto merged
myisam/sort.c:
Auto merged
mysql-test/r/func_sapdb.result:
Auto merged
mysql-test/r/variables.result:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/field.cc:
Auto merged
sql/ha_innodb.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_timefunc.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
tests/mysql_client_test.c:
Auto merged
mysql-test/r/type_datetime.result:
manual merge
mysql-test/r/type_decimal.result:
manual merge
mysql-test/t/type_datetime.test:
manual merge
mysql-test/t/type_decimal.test:
manual merge
sql/item.cc:
manual merge
myisam/mi_write.c:
type conversion fixed
myisam/sort.c:
type conversion fixed
sql/ha_federated.cc:
type conversion fixed
sql/ha_heap.cc:
type conversion fixed
sql/ha_innodb.cc:
type conversion fixed
sql/ha_myisam.cc:
type conversion fixed
sql/opt_range.cc:
type conversion fixed
sql/sql_map.cc:
type conversion fixed
sql/sql_select.cc:
type conversion fixed
sql/sql_update.cc:
type conversion fixed
CPUs / Intel's ICC compile
The bug is a combination of two problems:
1. IA64/ICC MySQL binaries use glibc's qsort(), not the one in mysys.
2. The order relation implemented by join_tab_cmp() is not transitive,
i.e. it is possible to choose such a, b and c that (a < b) && (b < c)
but (c < a). This implies that result of a sort using the relation
implemented by join_tab_cmp() depends on the order in which
elements are compared, i.e. the result is implementation-specific. Since
choose_plan() uses qsort() to pre-sort the
join tables using join_tab_cmp() as a compare function, the results of
the sorting may vary depending on qsort() implementation.
It is neither possible nor important to implement a better ordering
algorithm in join_tab_cmp(). Therefore the only way to fix it is to
force our own qsort() to be used by renaming it to my_qsort(), so we don't depend
on linker to decide that.
This patch also "fixes" bug #20530: qsort redefinition violates the
standard.
include/my_sys.h:
Renamed qsort() and qsort2() to my_qsort() and my_qsort2(). Since
previously we relied on stdlib.h to provide a declaration for qsort(), a
separate declaration for my_qsort() is now required.
libmysql/Makefile.shared:
Added mf_qsort.c to libmysql, since my_lib.c now uses my_qsort() instead of qsort().
myisam/ft_boolean_search.c:
Replaced qsort2() with my_qsort2().
myisam/ft_nlq_search.c:
Replaced qsort2() with my_qsort2().
myisam/myisampack.c:
Replaced qsort() with my_qsort().
myisam/sort.c:
Replaced qsort2() with my_qsort2().
mysys/mf_keycache.c:
Replaced qsort() with my_qsort().
mysys/mf_qsort.c:
Renamed qsort() to my_qsort() and qsort2() to my_qsort2().
mysys/mf_sort.c:
Replaced qsort2() with my_qsort2().
mysys/my_lib.c:
Replaced qsort() with my_qsort().
mysys/queues.c:
Replaced qsort2() with my_qsort2().
sql/item_cmpfunc.cc:
Replaced qsort2() with my_qsort2().
sql/item_cmpfunc.h:
Replaced qsort2() with my_qsort2().
sql/opt_range.cc:
Replaced qsort() with my_qsort().
sql/records.cc:
Replaced qsort() with my_qsort().
sql/sql_acl.cc:
Replaced qsort() with my_qsort().
sql/sql_array.h:
Replaced qsort() with my_qsort().
sql/sql_help.cc:
Replaced qsort() with my_qsort().
sql/sql_select.cc:
Replaced qsort() with my_qsort().
sql/examples/ha_tina.cc:
Replaced qsort() with my_qsort().
sql/sql_table.cc:
Replaced qsort() with my_qsort().
into polly.(none):/home/kaa/src/maint/mysql-5.0-maint
myisam/sort.c:
Auto merged
mysql-test/r/repair.result:
Auto merged
mysql-test/t/repair.test:
Auto merged
myisam_sort_buffer_size.
An incorrect length of the sort buffer was used when calculating the
maximum number of keys. When myisam_sort_buffer_size is small enough,
this could result in the number of keys < number of
BUFFPEK structures which in turn led to use of uninitialized BUFFPEKs.
Fixed by correcting the buffer length calculation.
myisam/sort.c:
Use a correct buffer length when calculating the maximum number of keys.
Assert that for each BUFFPEK structure there is at least one
corresponding key. Otherwise we would fail earlier and not reach
merge_buffers().
mysql-test/r/repair.result:
Added a test case for bug #31174.
mysql-test/t/repair.test:
Added a test case for bug #31174.
into mysql.com:/home/hf/work/30286/my50-30286
myisam/rt_index.c:
Auto merged
myisam/rt_mbr.c:
Auto merged
mysql-test/t/gis-rtree.test:
Auto merged
mysql-test/r/gis-rtree.result:
merging
As the result of DOUBLE claculations can be bigger
than DBL_MAX constant we use in code, we shouldn't use this constatn
as a biggest possible value.
Particularly the rtree_pick_key function set 'min_area= DBL_MAX' relying
that any rtree_area_increase result will be less so we return valid
key. Though in rtree_area_increase function we calculate the area
of the rectangle, so the result can be 'inf' if the rectangle is
huge enough, which is bigger than DBL_MAX.
Code of the rtree_pick_key modified so we always return a valid key.
myisam/rt_index.c:
Bug #30286 spatial index cause corruption and server crash!
always set the best_key with the first key we get, so we always return
somthing valid.
myisam/rt_mbr.c:
Bug #30286 spatial index cause corruption and server crash!
function comment extended
mysql-test/r/gis-rtree.result:
Bug #30286 spatial index cause corruption and server crash!
test result
mysql-test/t/gis-rtree.test:
Bug #30286 spatial index cause corruption and server crash!
test case
Several adjustments to make client libraries pass the link test
on both win32 and winx64, Visual Studio 2003 and 2005 (bug#30118)
win/README:
- Removed references to PARTITION engine, 5.1 only
win/configure.js:
- Removed references to PARTITION engine, 5.1 only
extra/CMakeLists.txt:
Use the special 'debug' list element to mark that "dbug" library
is only to be linked against if build type "Debug".
myisam/CMakeLists.txt:
Use the special 'debug' list element to mark that "dbug" library
is only to be linked against if build type "Debug".
scripts/CMakeLists.txt:
Use the special 'debug' list element to mark that "dbug" library
is only to be linked against if build type "Debug".
server-tools/instance-manager/CMakeLists.txt:
Use the special 'debug' list element to mark that "dbug" library
is only to be linked against if build type "Debug".
sql/CMakeLists.txt:
Use the special 'debug' list element to mark that "dbug" library
is only to be linked against if build type "Debug".
mysys/CMakeLists.txt:
Restored include path to "mysys" itself
dbug/CMakeLists.txt:
Changed to optionally be included to give a file list only
extra/yassl/CMakeLists.txt:
Changed to optionally be included to give a file list only
extra/yassl/taocrypt/CMakeLists.txt:
Changed to optionally be included to give a file list only
zlib/CMakeLists.txt:
Changed to optionally be included to give a file list only
libmysql/CMakeLists.txt:
For compatibility with Visual Studio 2005, list all files that are to
be part of the library build, i.e. libraries can't be built from other
libraries. Set SOURCE_SUBLIBS and include the file listings from
"zlib", "dbug", "taocrypt" and "yassl"
When using concurrent insert with parallel index reads, it could
happen that reading sessions found keys that pointed to records
yet to be written to the data file. The result was a report of
a corrupted table. But it was false alert.
When inserting a record in a table with indexes, the keys are
inserted into the indexes before the record is written to the data
file. When the insert happens concurrently to selects, an
index read can find a key that references the record that is not
yet written to the data file. To avoid any access to such record,
the select saves the current end of file position when it starts.
Since concurrent inserts are always appended at end of the data
file, the select can easily ignore any concurrently inserted record.
The problem was that the ignore was only done for non-exact key
searches (partial key or using >, >=, < or <=).
The fix is to ignore concurrently inserted records also for
exact key searches.
No test case. Concurrent inserts cannot be tested with the test
suite. Test cases are attached to the bug report.
myisam/mi_rkey.c:
Bug#29838 - myisam corruption using concurrent select ... and update
Fixed mi_rkey() to always ignore records beyond saved eof.
INSERT/DELETE/UPDATE followed by ALTER TABLE within LOCK TABLES
may cause table corruption on Windows.
That happens because ALTER TABLE writes outdated shared state
info into index file.
Fixed by removing obsolete workaround.
Affects MyISAM tables on Windows only.
myisam/mi_extra.c:
On windows when mi_extra(HA_EXTRA_PREPARE_FOR_DELETE) is called,
we release external lock and close index file. If we're in LOCK
TABLES, MyISAM state info doesn't get updated until UNLOCK TABLES.
That means when we release external lock and we're in LOCK TABLES,
we may write outdated state info.
As SQL layer closes all table instances, we do not need this
workaround anymore.
mysql-test/r/alter_table.result:
A test case for BUG#29957.
mysql-test/t/alter_table.test:
A test case for BUG#29957.
By default MyISAM overwrites .MYD and .MYI files no
DATA DIRECTORY option is used. This can lead to two tables
using the same .MYD and .MYI files (that can't be dropped).
To prevent CREATE TABLE from overwriting a file a new option
is introduced : keep_files_on_create
When this is on the CREATE TABLE throws an error if either
the .MYD or .MYI exists for a MyISAM table.
The option is off by default (resulting in compatible behavior).
include/my_base.h:
Bug #29325: introduce keep_files_on_create
myisam/mi_create.c:
Bug #29325: introduce keep_files_on_create
mysql-test/r/create.result:
Bug #29325: test case
mysql-test/t/create.test:
Bug #29325: test case
sql/ha_myisam.cc:
Bug #29325: introduce keep_files_on_create
sql/set_var.cc:
Bug #29325: introduce keep_files_on_create
sql/sql_class.h:
Bug #29325: introduce keep_files_on_create
sql/sql_table.cc:
Bug #29325: introduce keep_files_on_create
sql/unireg.cc:
Bug #29325: introduce keep_files_on_create
