Commit graph

1531 commits

Author SHA1 Message Date
Bradley C. Kuszmaul
59a8f06e93 The brtloader is now at 95.03% lines covered and 84.64% of branches taken. That's pretty good. We can work to remove a few more, but it's diminishing returns from here. [t:2647] Refs #2647.
git-svn-id: file:///svn/toku/tokudb@20537 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Bradley C. Kuszmaul
c4405b714c Make the ...dbufio tests better. Refs #2633. [t:2633]
git-svn-id: file:///svn/toku/tokudb@20534 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Dave Wells
2ef2752ced cleanup merge_some_files error path [t:2642]
git-svn-id: file:///svn/toku/tokudb@20533 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
1bba1e63b2 closes[t:2644] combine the brtloader panic and error callback state
git-svn-id: file:///svn/toku/tokudb@20526 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Barry Perlman
39670ab94b Allow logger status with no logger.
git-svn-id: file:///svn/toku/tokudb@20495 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
418b60fbd7 refs[t:2643] compile the brtloader tests
git-svn-id: file:///svn/toku/tokudb@20489 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Dave Wells
b3800e0177 fix loader_open test failures, closes #2591 [t:2591]
git-svn-id: file:///svn/toku/tokudb@20486 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
7b435f245a Fix at least one problem found by the loader-cleanup-test. Make the valgrind version of loader-cleanup-test use {{{-r 1000}}} so it runs faster. Refs #2642. [t:2642]
git-svn-id: file:///svn/toku/tokudb@20482 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Dave Wells
faea32e1ca add asserts, cleanup fta init
git-svn-id: file:///svn/toku/tokudb@20471 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
d567a309ac Fix #2643. (Make progress reporting work right again in the loader). close[t:2643]
git-svn-id: file:///svn/toku/tokudb@20470 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
af9d10bcd2 refs[t:2639] execute error return from kv_malloc
git-svn-id: file:///svn/toku/tokudb@20467 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
6c54d4ae28 closes[t:2639] recover from translation table realloc failures in the allocate block function of the brtloader. merge -r 20450:head from tokudb.2639
git-svn-id: file:///svn/toku/tokudb@20466 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
227abb8536 Fix #2533. (Avoid runts and don't make maximal fanout in loader). close[t:2533]
git-svn-id: file:///svn/toku/tokudb@20459 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
a6e22aadac Merge in the 2623 branch to main. Closes #2623. close[t:2623]
{{{
svn merge -r 20428:20432 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20433 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
5e03a29540 refs[t:2603] run some additional extractor tests
git-svn-id: file:///svn/toku/tokudb@20431 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
7fd7b5c15f closes[t:2634] fix some file open and close errors paths in the brtloader
git-svn-id: file:///svn/toku/tokudb@20430 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
47c5441a8d Merge the basic dbufio working test in. It doesn't test destination files, and the failure cases aren't running yet. But the ...dbufio() code looks fairly well covered anyway. [t:2623] Refs #2623.
{{{
svn merge -r 20423:20425 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20426 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
48b56ff14f refs[t:2603] inject errors from the poll callback into the extractor
git-svn-id: file:///svn/toku/tokudb@20424 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
53cb67c979 Merge some tests of error paths back to main line. Refs #2623. [t:2623]
{{{
svn merge -r 20415:20420 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20421 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
d3a0eca073 closes[t:2617] change some MALLOCs with no error checks to XMALLOCs
git-svn-id: file:///svn/toku/tokudb@20419 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
2651b61fd3 refs[t:2591] a better brtloader open test
git-svn-id: file:///svn/toku/tokudb@20418 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
661106bce3 Merge some changes back to main line (Refs #2623, #2631). [t:2623] [t:2631]
{{{
svn merge -r 20381:20412 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20413 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
0d184b5625 Fix #2630. close[t:2630]
git-svn-id: file:///svn/toku/tokudb@20405 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
da1467e398 do not define toku_merge_some_files_using_dbufio as a cilk function
git-svn-id: file:///svn/toku/tokudb@20403 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Yoni Fogel
7eb67d825b Windows port: ENODATA->TOKUDB_NO_DATA, disable unreachable code warning, disabled trace_mem for windows
git-svn-id: file:///svn/toku/tokudb@20400 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
127f1f927b refs[t:2626] fix some void* type conversions
git-svn-id: file:///svn/toku/tokudb@20389 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
d009c784dc closes[t:2626] fix the brtloader add_row realloc error recovery
git-svn-id: file:///svn/toku/tokudb@20385 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
9b57707611 Closes #2498 [t:2498] Made DICTIONARY_ID 64 bits.
git-svn-id: file:///svn/toku/tokudb@20383 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
5e75d244c1 Merge #2623 changes onto main line. The new test fails, but I need to get a merge from main again. Refs #2623. [t:2623]
{{{
svn merge -r 20348:20378 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20379 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
6f5c9f2fd4 Use {{{size_factor}}} to size rowsets. Fixes #2628. close[t:2628]
git-svn-id: file:///svn/toku/tokudb@20375 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
2108cc42c6 Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20364 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
8db89b8438 closes[t:2625] move the new assert macros to the toku_assert.h
git-svn-id: file:///svn/toku/tokudb@20353 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
001f15c2f5 closes[t:2622] when realloc fails in the brtloader writer, mark the dbuf in error and check the error later
git-svn-id: file:///svn/toku/tokudb@20350 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
d6dc7aad45 Change a MALLOC to an XMALLOC. Refs #2623. [t:2623]
git-svn-id: file:///svn/toku/tokudb@20347 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
0b177d642b Merge the memory management into the main line. Refs #2613. [t:2613].
{{{
svn merge -r 20329:20340 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20341 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
7b852a3f96 remove an invariant to make the code work
git-svn-id: file:///svn/toku/tokudb@20337 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
c9d3fb0b97 more lazy_assert
git-svn-id: file:///svn/toku/tokudb@20336 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
d995208a1c remove lazy_assert in loader_do_put
git-svn-id: file:///svn/toku/tokudb@20335 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
366941f3cd fix leak with primary_rowset [t:2591]
git-svn-id: file:///svn/toku/tokudb@20334 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
b611db04c2 cleanup error handling in toku_brt_loader_open, toku_loader_open
git-svn-id: file:///svn/toku/tokudb@20331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
27cbb4c70a closes[t:2620] cleanup the lazy assert in the brtloader's close and abort functions
git-svn-id: file:///svn/toku/tokudb@20330 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
460ef9b4ca scale back the extractor error tests refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20321 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
abac9d89b3 refs[t:2603] run the brtloader tests in their own directories
git-svn-id: file:///svn/toku/tokudb@20310 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
396aff4100 closes[t:2616] fix the file open fail path in the loader_do_i function
git-svn-id: file:///svn/toku/tokudb@20308 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
432e750814 refs[t:2613] get the brtloader tests working with the new memory accounting
git-svn-id: file:///svn/toku/tokudb@20307 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
a59cb89830 change some lazy_asserts refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20306 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
9e741558e2 test error from generate key callback refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20305 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Bradley C. Kuszmaul
e3d629fb2e Merge in memory management for the extractor thread. Refs #2613. [t:2613].
{{{
svn merge -r 20278:20301 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20302 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
11897a5141 closes[t:2588] fix the loader create close test
git-svn-id: file:///svn/toku/tokudb@20298 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Dave Wells
f0b095543d assert labeling 3
git-svn-id: file:///svn/toku/tokudb@20296 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00