Analysis:
---------
When the server is out of memory, an error is raised
to indicate the same. Handling the error requires
more memory to be allocated which fails, hence the
error handling loops in a recursion and causes the
server to crash.
Fix:
---
a) Prevents pushing the 'out of memory' error condition
to the diagnostic area as it requires memory allocation.
GET DIAGNOSTICS, SHOW WARNINGS and SHOW ERRORS statements
will not show information about this error. However the
'out of memory' error is returned to the client.
b) It sets the ME_FATALERROR flag when 'out of memory' errors
are reported (for places where the flag is not already set).
This flag prevents activation of SP error handlers which also
require memory allocation and therefore are likely to fail.
Problem:-
In case of blob data field, UNION ALL doesn't give correct result.
Analysis:-
In MyISAM table, when we dont want to check for the distinct for particular
key, we set the key_map to zero.
While writing record in MyISAM table, we check the distinct with the help
of keys, by checking whether that key is active in key_map and then writing
the record.
In case of blob field, we are checking for distinct by unique constraint,
where we are not checking whether that unique key is active or not in key_map.
Solution:
Before checking for distinct, check whether any key is active in key_map.
storage/myisam/mi_write.c:
check whether key_map is active before checking distinct.
Problem:-
In case of blob data field, UNION ALL doesn't give correct result.
Analysis:-
In MyISAM table, when we dont want to check for the distinct for particular
key, we set the key_map to zero.
While writing record in MyISAM table, we check the distinct with the help
of keys, by checking whether that key is active in key_map and then writing
the record.
In case of blob field, we are checking for distinct by unique constraint,
where we are not checking whether that unique key is active or not in key_map.
Solution:-
Before checking for distinct, check whether any key is active in key_map.
storage/myisam/mi_write.c:
check whether key_map is active before checking distinct.
The patch lifts the limitation of the current implementation
of ALTER TABLE that does not allow to build unique/primary
indexes by sort for MyISAM and Aria engines.
Problem description: Incorrect key file. Key file is corrupted,
while reading the keys from the file. The problem here is that
keyseg->start (which should point to the beginning of a field)
is pointing beyond total record length.
Fix: If keyseg->start is greater than total record length then
return error.
Problem description: Corrupt key file for the table. Size of the
key is greater than the maximum specified size. This results in
the overflow of the key buffer while reading the key from key
file.
Fix: If size of key is greater than the maximum size it returns
an error before writing it into the key buffer. Gives error as
corrupt file but no stack overflow.
Problem description:
mysql server crashes when we run repair table on currupted table.
Analysis:
The problem with this bug seem to be key_reflength out of bounds
(186 according to debugger). We read this value from meta-data
segment of .MYI file while doing mi_open().
If you look into _mi_kpointer() you can see that the upper limit
for key_reflength is 7.
Solution:
In mi_open() there is a line like:
if (share->base.keystart > 65535 || share->base.rec_reflength > 8)
we should verify key_reflength here as well.
TO 'MYISAM_SORT_BUFFER_SIZE'
Problem: 'myisam_sort_buffer_size' is a parameter used by
mysqld program only whereas 'sort_buffer_size' is used by
mysqld and myisamchk programs. But the error message printed
when myisamchk program is run with insufficient buffer size
is myisam_sort_buffer_size is too small which may mislead to the
server parameter myisam_sort_buffer_size.
SOLUTION: A parameter 'myisam_sort_buffer_size' is added as an
alias for 'sort_buffer_size' and the 'sort_buffer_size' parameter
is marked as deprecated. So myisamchk also has both the parameters
with the same role.
TO 'MYISAM_SORT_BUFFER_SIZE'
Problem: 'myisam_sort_buffer_size' is a parameter used by
mysqld program only whereas 'sort_buffer_size' is used by
mysqld and myisamchk programs. But the error message printed
when myisamchk program is run with insufficient buffer size
is myisam_sort_buffer_size is too small which may mislead to the
server parameter myisam_sort_buffer_size.
SOLUTION: A parameter 'myisam_sort_buffer_size' is added as an
alias for 'sort_buffer_size' and the 'sort_buffer_size' parameter
is marked as deprecated. So myisamchk also has both the parameters
with the same role.
Generalized support for auto-updated and/or auto-initialized timestamp
and datetime columns. This patch is a reimplementation of MySQL's
"WL#5874: CURRENT_TIMESTAMP as DEFAULT for DATETIME columns". In order to
ease future merges, this implementation reused few function and variable
names from MySQL's patch, however the implementation is quite different.
TODO:
The only unresolved problem in this patch is the semantics of LOAD DATA for
TIMESTAMP and DATETIME columns in the cases when there are missing or NULL
columns. I couldn't fully comprehend the logic behind MySQL's behavior and
its relationship with their own documentation, so I left the results to be
more consistent with all other LOAD cases.
The problematic test cases can be seen by running the test file function_defaults,
and observing the test case differences. Those were left on purpose for discussion.
and small collateral changes
mysql-test/lib/My/Test.pm:
somehow with "print" we get truncated writes sometimes
mysql-test/suite/perfschema/r/digest_table_full.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/dml_handler.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/information_schema.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/nesting.result:
this differs, because we don't rewrite general log queries, and multi-statement
packets are logged as a one entry. this result file is identical to what mysql-5.6.5
produces with the --log-raw option.
mysql-test/suite/perfschema/r/relaylog.result:
MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB.
mysql-test/suite/perfschema/r/server_init.result:
MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup.
mysql-test/suite/perfschema/r/stage_mdl_global.result:
this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not
do that, and this causes useless mutex locks and waits.
mysql-test/suite/perfschema/r/statement_digest.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_consumers.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_long_query.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result:
will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged
two tests still fail:
main.innodb_icp and main.range_vs_index_merge_innodb
call records_in_range() with both range ends being open
(which triggers an assert)
- index_merge/intersection is unable to work on GIS indexes, because:
1. index scans have no Rowid-Ordered-Retrieval property
2. When one does an index-only read over a GIS index, they do not
get the index tuple, because index only contains bounding box of the geometry.
This is why key_copy() call crashed.
This patch fixes#1, which makes the problem go away. Theoretically, it would
be nice to check #2, too, but SE API semantics is not sufficiently precise to do it.
ISSUE: Incorrect key file. Key file is corrupted,
Reading incorrect key information (keyseg)
from index file. Key definition in .MYI
and .FRM file differs. Starting pointer
to read the keyseg information is changed
to a value greater than the pack_reclength.
Memcpy tries to read keyseg information from
unallocated memory which causes the crash.
SOLUTION: One more check added to compare the
the key definition in .MYI and .FRM
file. If the definition differ, server
produces an error.
Optimizator fails using index with ST_Within(g, constant_poly).
per-file comments:
mysql-test/r/gis-rt-precise.result
test result fixed.
mysql-test/r/gis-rtree.result
test result fixed.
mysql-test/suite/maria/r/maria-gis-rtree-dynamic.result
test result fixed.
mysql-test/suite/maria/r/maria-gis-rtree-trans.result
test result fixed.
mysql-test/suite/maria/r/maria-gis-rtree.result
test result fixed.
storage/maria/ma_rt_index.c
Use MBR_INTERSECT mode when optimizing the select WITH ST_Within.
storage/myisam/rt_index.c
Use MBR_INTERSECT mode when optimizing the select WITH ST_Within.
FULLTEXT INDEX AND CONCURRENT DML.
Problem Statement:
------------------
1) Create a table with FT index.
2) Enable concurrent inserts.
3) In multiple threads do below operations repeatedly
a) truncate table
b) insert into table ....
c) select ... match .. against .. non-boolean/boolean mode
After some time we could observe two different assert core dumps
Analysis:
--------
1)assert core dump at key_read_cache():
Two select threads operating in-parallel on same key
root block.
1st select thread block->status is set to BLOCK_ERROR
because the my_pread() in read_block() is returning '0'.
Truncate table made the index file size as 1024 and pread
was asked to get the block of count bytes(1024 bytes)
from offset of 1024 which it cannot read since its
"end of file" and retuning '0' setting
"my_errno= HA_ERR_FILE_TOO_SHORT" and the key_file_length,
key_root[0] is same i.e. 1024. Since block status has BLOCK_ERROR
the 1st select thread enter into the free_block() and will
be under wait on conditional mutex by making status as
BLOCK_REASSIGNED and goes for wait_on_readers(). Other select
thread will also work on the same block and sees the status as
BLOCK_ERROR and enters into free_block(), checks for BLOCK_REASSIGNED
and asserting the server.
2)assert core dump at key_write_cache():
One select thread and One insert thread.
Select thread gets the unlocks the 'keycache->cache_lock',
which allows other threads to continue and gets the pread()
return value as'0'(please see the explanation above) and
tries to get the lock on 'keycache->cache_lock' and waits
there for the lock.
Insert thread requests for the block, block will be assigned
from the hash list and makes the page_status as
'PAGE_WAIT_TO_BE_READ' and goes for the read_block(), waits
in the queue since there are some other threads performing
reads on the same block.
Select thread which was waiting for the 'keycache->cache_lock'
mutex in the read_block() will continue after getting the my_pread()
value as '0' and sets the block status as BLOCK_ERROR and goes to
the free_block() and go to the wait_for_readers().
Now the insert thread will awake and continues. and checks
block->status as not BLOCK_READ and it asserts.
Fix:
---
In the full text code, multiple readers of index file is not guarded.
Hence added below below code in _ft2_search() and walk_and_match().
to lock the key_root I have used below code in _ft2_search()
if (info->s->concurrent_insert)
mysql_rwlock_rdlock(&share->key_root_lock[0]);
and to unlock
if (info->s->concurrent_insert)
mysql_rwlock_unlock(&share->key_root_lock[0]);
storage/myisam/ft_boolean_search.c:
Since its a recursion function, to avoid confusion in taking and
releasing the locks, renamed _ft2_search() to _ft2_search_internal()
function. And _ft2_search() will take the lock, call
_ft2_search_internal() and release the lock in case of concurrent
inserts.
storage/myisam/ft_nlq_search.c:
Added read locks code in walk_and_match()
