Commit graph

3245 commits

Author SHA1 Message Date
Yoni Fogel
4d0f2e5773 [t:2624] fix windows compile of r20449 (off_t->toku_off_t)
git-svn-id: file:///svn/toku/tokudb@20522 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Barry Perlman
39670ab94b Allow logger status with no logger.
git-svn-id: file:///svn/toku/tokudb@20495 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
418b60fbd7 refs[t:2643] compile the brtloader tests
git-svn-id: file:///svn/toku/tokudb@20489 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Bradley C. Kuszmaul
bfe190f763 #2642 runs with -r 1000 and no valgrind problems. The problem is we must actually call fclose even when injecting errors in bad_fclose. [t:2642] Refs #2642
git-svn-id: file:///svn/toku/tokudb@20487 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Dave Wells
b3800e0177 fix loader_open test failures, closes #2591 [t:2591]
git-svn-id: file:///svn/toku/tokudb@20486 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
7b435f245a Fix at least one problem found by the loader-cleanup-test. Make the valgrind version of loader-cleanup-test use {{{-r 1000}}} so it runs faster. Refs #2642. [t:2642]
git-svn-id: file:///svn/toku/tokudb@20482 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Dave Wells
faea32e1ca add asserts, cleanup fta init
git-svn-id: file:///svn/toku/tokudb@20471 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
d567a309ac Fix #2643. (Make progress reporting work right again in the loader). close[t:2643]
git-svn-id: file:///svn/toku/tokudb@20470 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
af9d10bcd2 refs[t:2639] execute error return from kv_malloc
git-svn-id: file:///svn/toku/tokudb@20467 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
6c54d4ae28 closes[t:2639] recover from translation table realloc failures in the allocate block function of the brtloader. merge -r 20450:head from tokudb.2639
git-svn-id: file:///svn/toku/tokudb@20466 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
227abb8536 Fix #2533. (Avoid runts and don't make maximal fanout in loader). close[t:2533]
git-svn-id: file:///svn/toku/tokudb@20459 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Barry Perlman
bc8ec47a33 [t:2621] [t:2624] Port file system wrappers to Windows.
git-svn-id: file:///svn/toku/tokudb@20449 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
a6e22aadac Merge in the 2623 branch to main. Closes #2623. close[t:2623]
{{{
svn merge -r 20428:20432 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20433 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
5e03a29540 refs[t:2603] run some additional extractor tests
git-svn-id: file:///svn/toku/tokudb@20431 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
7fd7b5c15f closes[t:2634] fix some file open and close errors paths in the brtloader
git-svn-id: file:///svn/toku/tokudb@20430 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Barry Perlman
8b7ecdf800 [t:2499] loader-cleanup-test: Improve error reporting
git-svn-id: file:///svn/toku/tokudb@20429 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
47c5441a8d Merge the basic dbufio working test in. It doesn't test destination files, and the failure cases aren't running yet. But the ...dbufio() code looks fairly well covered anyway. [t:2623] Refs #2623.
{{{
svn merge -r 20423:20425 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20426 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
48b56ff14f refs[t:2603] inject errors from the poll callback into the extractor
git-svn-id: file:///svn/toku/tokudb@20424 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
53cb67c979 Merge some tests of error paths back to main line. Refs #2623. [t:2623]
{{{
svn merge -r 20415:20420 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20421 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
d3a0eca073 closes[t:2617] change some MALLOCs with no error checks to XMALLOCs
git-svn-id: file:///svn/toku/tokudb@20419 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
2651b61fd3 refs[t:2591] a better brtloader open test
git-svn-id: file:///svn/toku/tokudb@20418 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Dave Wells
02e2cc1595 reverse polarity of -M option - now default behavior is use half of memory for cachesize
git-svn-id: file:///svn/toku/tokudb@20417 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
661106bce3 Merge some changes back to main line (Refs #2623, #2631). [t:2623] [t:2631]
{{{
svn merge -r 20381:20412 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20413 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
0d184b5625 Fix #2630. close[t:2630]
git-svn-id: file:///svn/toku/tokudb@20405 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
da1467e398 do not define toku_merge_some_files_using_dbufio as a cilk function
git-svn-id: file:///svn/toku/tokudb@20403 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Yoni Fogel
7eb67d825b Windows port: ENODATA->TOKUDB_NO_DATA, disable unreachable code warning, disabled trace_mem for windows
git-svn-id: file:///svn/toku/tokudb@20400 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Yoni Fogel
5539b538a9 Fix windows build
git-svn-id: file:///svn/toku/tokudb@20399 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
7e9e6af90a [t:2499] loader-cleanup-test: finer control over verbosity.
git-svn-id: file:///svn/toku/tokudb@20391 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
127f1f927b refs[t:2626] fix some void* type conversions
git-svn-id: file:///svn/toku/tokudb@20389 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
d009c784dc closes[t:2626] fix the brtloader add_row realloc error recovery
git-svn-id: file:///svn/toku/tokudb@20385 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
9b57707611 Closes #2498 [t:2498] Made DICTIONARY_ID 64 bits.
git-svn-id: file:///svn/toku/tokudb@20383 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
5e75d244c1 Merge #2623 changes onto main line. The new test fails, but I need to get a merge from main again. Refs #2623. [t:2623]
{{{
svn merge -r 20348:20378 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20379 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
6f5c9f2fd4 Use {{{size_factor}}} to size rowsets. Fixes #2628. close[t:2628]
git-svn-id: file:///svn/toku/tokudb@20375 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
3da94d6415 [t:2499] Use correct LOADER_USE_PUTS flag. Add engine status output when verbose>=2.
git-svn-id: file:///svn/toku/tokudb@20374 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
2108cc42c6 Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20364 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
9ccf76833d Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20360 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
939a4ea0c5 [t:2499] Add ability to control size_factor and count temp files.
git-svn-id: file:///svn/toku/tokudb@20359 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
8db89b8438 closes[t:2625] move the new assert macros to the toku_assert.h
git-svn-id: file:///svn/toku/tokudb@20353 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
001f15c2f5 closes[t:2622] when realloc fails in the brtloader writer, mark the dbuf in error and check the error later
git-svn-id: file:///svn/toku/tokudb@20350 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
d6dc7aad45 Change a MALLOC to an XMALLOC. Refs #2623. [t:2623]
git-svn-id: file:///svn/toku/tokudb@20347 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
0b177d642b Merge the memory management into the main line. Refs #2613. [t:2613].
{{{
svn merge -r 20329:20340 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20341 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
7b852a3f96 remove an invariant to make the code work
git-svn-id: file:///svn/toku/tokudb@20337 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
c9d3fb0b97 more lazy_assert
git-svn-id: file:///svn/toku/tokudb@20336 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
d995208a1c remove lazy_assert in loader_do_put
git-svn-id: file:///svn/toku/tokudb@20335 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
366941f3cd fix leak with primary_rowset [t:2591]
git-svn-id: file:///svn/toku/tokudb@20334 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
b611db04c2 cleanup error handling in toku_brt_loader_open, toku_loader_open
git-svn-id: file:///svn/toku/tokudb@20331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
27cbb4c70a closes[t:2620] cleanup the lazy assert in the brtloader's close and abort functions
git-svn-id: file:///svn/toku/tokudb@20330 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
62d107b6d8 refs[t:2606] ship the cilk rts
git-svn-id: file:///svn/toku/tokudb@20323 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
460ef9b4ca scale back the extractor error tests refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20321 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
abac9d89b3 refs[t:2603] run the brtloader tests in their own directories
git-svn-id: file:///svn/toku/tokudb@20310 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00