A big test was written and is committed, which found 3 bugs in total:
- ALTER TABLE PAGE_CHECKSUM=0 sometimes had no effect
- ALTER TABLE ENGINE=MARIA sometimes changed page checksumming in
the table
- SHOW CREATE TABLE and 'maria_chk -dv' disagreed on the presence
of page checksumming.
They are all fixed here. Side-effect is that SHOW CREATE TABLE now
always prints a PAGE_CHECKSUM clause for Maria tables.
mysql-test/mysql-test-run.pl:
allow calling maria_chk and maria_pack in tests
mysql-test/r/maria.result:
PAGE_CHECKSUM=0 is now always printed
mysql-test/t/create.test:
PAGE_CHECKSUM= is now always present in SHOW CREATE TABLE of Maria tables
mysql-test/t/disabled.def:
better bug number
sql/sql_table.cc:
If ALTER TABLE PAGE_CHECKSUM=#, it affects the engine's data (structure
of data pages) so a full table rebuild is needed. We already did
so for ROW_FORMAT=#, following same logic. This fixes disagreements
between SHOW CREATE TABLE and 'maria_chk -dv' regarding the
presence of page checksums after certain ALTER TABLE (as seen
with the attached testcase).
storage/maria/ha_maria.cc:
In ALTER TABLE PAGE_CHECKSUM=0, ha_maria::update_create_info() started
with create_info->page_checksum=HA_CHOICE_NO and wrongly set it
to the table's original setting, which is HA_CHOICE_YES if the table
had page checksums, in which case the ALTER left page checksum
in the table.
The fix for this bug is: only if create_info->page_checksum is
undefined (no clause in the ALTER TABLE, or we are in SHOW CREATE TABLE)
we may set HA_CHOICE_YES.
The second bug in this file was that the code left HA_CHOICE_UNDEF if
the table didn't have page checksums originally, leading ALTER TABLE
ENGINE=MARIA to change the table's page checksum to the value of
maria_page_checksum.
This is fixed by setting create_info->page_checksum to HA_CHOICE_NO
if UNDEF and table does not have page checksum.
The side-effect of this last fix, because ha_maria::update_create_info()
is also called by SHOW CREATE TABLE, is that:
SET GLOBAL maria_page_checksum=0;
CREATE TABLE t(a INT) ENGINE=MARIA;
SHOW CREATE TABLE t;
which used to not show a PAGE_CHECKSUM= clause, now shows PAGE_CHECKSUM=0.
I consider this side-effect good:
- clearer for users: it eliminates the differences between the
above and this:
SET GLOBAL maria_page_checksum=0;
CREATE TABLE t(a INT) ENGINE=MARIA PAGE_CHECKSUM=0;
SHOW CREATE TABLE t;
which already showed PAGE_CHECKSUM=0; difference which is not easy
to explain.
- if using mysqldump to copy from one server to another, it eliminates
the dependency on the value of maria_page_checksum being the same on
original server and new server.
mysql-test/r/maria-page-checksum.result:
Result. If you undo the code fixes and run the test, the new result
file will show bugs at:
error lineno 56 : expected PAGE_CHECKSUM=1, got Page checksums are not used
error lineno 73 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 110 : expected PAGE_CHECKSUM=1, got Page checksums are not used
error lineno 164 : expected PAGE_CHECKSUM=1, got Page checksums are not used
error lineno 181 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 218 : expected PAGE_CHECKSUM=1, got Page checksums are not used
error lineno 253 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 307 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 361 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 415 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 488 : expected PAGE_CHECKSUM=1, got Page checksums are not used
error lineno 505 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 542 : expected PAGE_CHECKSUM=1, got Page checksums are not used
error lineno 577 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
error lineno 631 : expected PAGE_CHECKSUM=0, got ) ENGINE=MARIA DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1
(lineno is line number in the result file)
mysql-test/t/maria-page-checksum.test:
Test for the 3 bugs
when modifying R-tree indices we forgot to unpin pages (causing
flush at-end-of-statement to fail - because page is pinned - and thus
mark table corrupted) and also to unlock key_del (causing assertion
failure info->used_key_del==0 at maria_close()).
storage/maria/ma_rt_index.c:
When modifying pages of R-tree indices we keep them pinned, and
lock key_del, so we here make sure to unpin and unlock at the end
of the modification. I checked all functions which add to info->pinned_pages
or call _ma_lock_key_del(): the parent is always maria_rtree_insert()
or maria_rtree_delete(), which thus do the unpin/unlock now.
mysql-test/r/maria-gis-rtree.result:
new result, which is incorrect (filed as BUG#35351) but at least
test does not crash anymore
mysql-test/t/maria-gis-rtree.test:
test of Maria's GIS functionality (R-tree indices etc); copied from
MyISAM's gis-rtree.test; used to crash without the bugfixes.
storage/maria/ma_locking.c:
define 'share' only in block which needs it
storage/maria/unittest/ma_test_recovery.pl:
more straightforward way to compute file's MD5
%llu replaces with %s/llstr() as in other MySQL code.
storage/maria/ma_loghandler.c:
my_stat is not reliable under windows for determinating file
length so it is replaced with my_seek.
%llu replaces with %s/llstr() as in other MySQL code.
Problem was that page in bitmap was marked as full even if there was free places in page directory
mysql-test/r/maria.result:
Test case for problem with head/tail page with 255 entries
(Bug 35048 "maria table corruption reported when transactional=0)
mysql-test/t/maria.test:
Test case for problem with head/tail page with 255 entries
(Bug 35048 "maria table corruption reported when transactional=0)
storage/maria/ma_blockrec.c:
Fix to ensure that bitmap is marked 'full' when the head/tail page directory is full
storage/maria/ma_check.c:
Better check when directory for head/tail pages are marked full (The page directory can't hold a row tail + blob tails)
if nothing was added to the log
(BUG#34712 maria_read_log changes log
file content on several platforms though not needed)
Fixed callback call (now under loghandler lock).
storage/maria/ma_loghandler.c:
Flag that signaling about "everythig-flushed" state of
the log added, to avoid changing log on flush request
if nothing was written but horizon placed on the page boarder.
Fixed callback call (now under loghandler lock).
the moment. (BUG#34634)
Fixed potential bug in pinning schema.
storage/maria/ma_pagecache.c:
Fixed problem of deleting blocks which are being evicted at
the moment.
Fixed potential bug in pinning schema.
Fixed crash when using other maria block size than 8192
Fixed bug that caused tables to be reported as crashed if maria_log_control was deleted
mysql-test/valgrind.supp:
Suppress leaks on SuSE 10.3, x86 64 bit
sql/mysqld.cc:
Don't give warning in case of timeout for pthread_cond_timedwait
Fixed crashing bug when starting mysqld with --dbug
storage/maria/ha_maria.cc:
Fixed crash when using other block size than 8192 (Crash happend later in bitmap page handling)
storage/maria/ma_locking.c:
When we write a new uuid for a zerofilled file, also update the lsn's
storage/maria/ma_open.c:
Don't update lsn on open. Wait until first time file is changed
Fixed bug that caused tables to be reported as crashed if maria_log_control was deleted
strings/strmake.c:
Fixed warnings for strings without end 0 when running under valgrind.
storage/maria/ha_maria.cc:
enable LOAD INDEX for Maria
storage/maria/ma_preload.c:
Making index preloading work for Maria.
storage/maria/maria_def.h:
We don't need this #define anymore, ma_test_recovery.pl uses
zerofill to do comparisons of physical files.
mysql-test/r/maria-preload.result:
result. The values of maria_pagecache_reads look too high, see
BUG#35030 "maria_pagecache_read* status variables updated at wrong moment"
mysql-test/t/maria-preload.test:
Test of index preloading in Maria
fixes for ma_test_recovery.pl to work in release builds too:
- bugfix in maria_zerofill_index()
- applying of LOGREC_INSERT_ROW_BLOBS now zeroes unused end of non-full
blob page (a mutation of tail page when it takes >75% of maria_block_size)
like write_full_pages() does.
storage/maria/ma_blockrec.c:
When we write a non-full blob page at run-time, we zero the rest of
it (see write_full_pages()). We now do the same in
_ma_apply_redo_insert_row_blobs(): this is consistent and helps
having log-applying produce the same page as run-time.
storage/maria/ma_check.c:
maria_zerofill_index() was wrong: it didn't zero certain bytes
because it believed that the count of relevant bytes is
_ma_get_page_used(share, buff) + share->keypage_header,
whereas it's only the first term.
storage/maria/ma_pagecache.c:
typo
storage/maria/ma_pagecache.h:
typo
storage/maria/maria_chk.c:
enable --zerofill-keep-lsn in all builds
storage/maria/unittest/ma_test_recovery.pl:
now even release builds can run ma_test_recovery.pl, because zerofill
makes tables of this test (made by run-time and by log-applying)
identical.
storage/maria/unittest/ma_test_recovery.pl:
Tell how many tables were zerofilled (just for information, a varying
number does not make the test fail).
Don't let --zerofill-keep-lsn influence next runs (which it shouldn't
do in theory anyway, but testing should not believe theory): restore
non-zerofilled tables right after comparison is finished.
in release builds, use it to fine-tune physical file comparisons in
ma_test_recovery.pl.
storage/maria/CMakeLists.txt:
build maria_dump_log under Windows too
storage/maria/maria_chk.c:
make --zerofill-keep-lsn usable in release builds too, it makes sense.
This option needn't reset state's LSNs, and it's more useful if it
does not (see added comment) as it changes less the table.
storage/maria/unittest/ma_test_recovery.pl:
- getting rid of 'cmp'
- if physical comparison of two MAD or MAI files fails, it might
be in irrelevant portions (unushed index or data page pieces):
we use --zerofill-keep-lsn and retry the comparison. This fixes
a part of test failures under Windows.
into 88-214-96-46.dialup.umc.net.ua:/Users/bell/mysql/bk/work-maria-test
storage/maria/ma_pagecache.c:
Auto merged
storage/maria/unittest/Makefile.am:
merge
storage/maria/unittest/ma_pagecache_consist.c:
merge
scheme for the same thread locking).
include/my_pthread.h:
Added variable for lock diffirentiation.
include/wqueue.h:
New release call.
mysys/wqueue.c:
New release call in case of read/write lock.
storage/maria/ma_pagecache.c:
Locking for read and write made waitimg for each other.
storage/maria/unittest/Makefile.am:
New test added.
storage/maria/unittest/ma_pagecache_consist.c:
Fixed thread initialization in the test.
storage/maria/unittest/ma_pagecache_rwconsist.c:
New BitKeeper file ``storage/maria/unittest/ma_pagecache_rwconsist.c''
strings/bmove512.c:
Fixed typo (Serious for systems that doesn't have longlong, but apparently we don't have theese...)
storage/maria/file_formats.txt:
Description of file formats (for now, only the header of the index file)
