Commit graph

301 commits

Author SHA1 Message Date
Bradley C. Kuszmaul
c49d7a8c0c The first loader gets 1/2 of cachetable memory.
* That's 2/3 of 3/4, which is calculated as 1/4 is unreservable.  The loader asks for 2/3 of what's reservable, getting one half.
The second loader gets 1/6 of cachetable memory.
 * That's 2/3 of 1/4: The loader asks for 2/3 of what's reservable.  1/2 has been reserved.  1/4 is unreservable.  1/4 is reservable.  2/3 of 1/4 is 1/6.
The third loader gets 1/18 of cachetable memory. 
 * That's 2/3 or 1/12.   (1/4 is unreservable. 1/2 was reserved by the first loader.  1/6 by the second.  That leaves 1/12)
Generally the next one gets 1/3 of what the previous one got.

Refs #2759. [t:2759]


git-svn-id: file:///svn/toku/tokudb@21564 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Bradley C. Kuszmaul
f635f4d458 Merge the fix for #2755 and #2759 to main. [t:2755] [t:2759]
{{{
svn merge -r21499:21502 https://svn.tokutek.com/tokudb/toku/tokudb.2755
}}}
.


git-svn-id: file:///svn/toku/tokudb@21503 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Rich Prohaska
c37567705f #2738 refs[t:2738] merge dbufio bug fix to main
git-svn-id: file:///svn/toku/tokudb@21260 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Rich Prohaska
76dcd3fe21 2738 refs[t:2738] test case for the double buffer hang
git-svn-id: file:///svn/toku/tokudb@21217 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Rich Prohaska
132926232d refs[t:2683] fix the newbrt/tests/Makefile rule 2b, should be 3a
git-svn-id: file:///svn/toku/tokudb@20857 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:18 -04:00
Bradley C. Kuszmaul
53b88b1fb1 Fix #2683. Make the big brtloader-test-extractor tests not use valgrind. close[t:2683]
git-svn-id: file:///svn/toku/tokudb@20849 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:18 -04:00
Yoni Fogel
6b16623e8a Fix windows compile.
Poison __sync_fetch_and_add and __sync_add_and_fetch, wrote toku wrappers and windows equivalents
fix bug in toku_sync_fetch_and_(in|de)crement_int32 where it returned result instead of original

git-svn-id: file:///svn/toku/tokudb@20848 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:17 -04:00
Rich Prohaska
e5ce7dfb2a refs[t:2681] omit the test that crashes cilkscreen
git-svn-id: file:///svn/toku/tokudb@20842 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:17 -04:00
Yoni Fogel
c9b2b066dd [t:2561] Merge tokudb.2561b to main. Disabled auto-upgrade and auto-upgrade tests
git-svn-id: file:///svn/toku/tokudb@20778 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:17 -04:00
Rich Prohaska
c5bad31abf closes[t:2656] run some cilkscreen tests
git-svn-id: file:///svn/toku/tokudb@20753 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
d694dbaf93 closes[t:2665] quiet logcursor-bad-checksum
git-svn-id: file:///svn/toku/tokudb@20722 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
9d695b7990 closes[t:2666] quiet recovery-cend-cbegin
git-svn-id: file:///svn/toku/tokudb@20721 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
85fbe869d1 closes[t:2667] quiet recovery-empty
git-svn-id: file:///svn/toku/tokudb@20719 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Bradley C. Kuszmaul
c74c406581 Fix #2664. (cachetable-scan should be quiet). close[t:2664]
git-svn-id: file:///svn/toku/tokudb@20713 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
4952a05f32 refs[t:2656] run some cilkscreen tests
git-svn-id: file:///svn/toku/tokudb@20693 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
9d292108f8 closes[t:2637] common brtloader error injector source
git-svn-id: file:///svn/toku/tokudb@20687 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
3f6fe2beac refs[t:2656] fix some data races in the brtloader tests
git-svn-id: file:///svn/toku/tokudb@20660 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
6d894d3f94 refs[t:2656] cilkscreen some brtloader-test-writer-errors
git-svn-id: file:///svn/toku/tokudb@20658 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
d86d1e5361 closes[t:2659] fix the data race in the finish_primary_rows_internal function. found by cilkscreen ./brtloader-test-extractor -s
git-svn-id: file:///svn/toku/tokudb@20657 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
650e2c0079 refs[t:2656] cilkscreen some brtloader tests
git-svn-id: file:///svn/toku/tokudb@20656 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
9e02290285 closes[t:2642] fix a mem leak in the brtloader found with the cleanup test
git-svn-id: file:///svn/toku/tokudb@20627 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
4ee75d917c closes[t:2650] test the brtloader extractor with various input rows
git-svn-id: file:///svn/toku/tokudb@20570 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
c62f8829e1 closes[t:2650] test the brtloader extractor with various input rows
git-svn-id: file:///svn/toku/tokudb@20569 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
6af1a32e4e refs[t:2650] verify the temp files created by the extractor
git-svn-id: file:///svn/toku/tokudb@20558 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Bradley C. Kuszmaul
d14326cc0e brtloader.c is now 96.61% of lines and 87.13% of branches. [t:2647] Refs #2647.
git-svn-id: file:///svn/toku/tokudb@20539 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
543dd95cdb closes[t:2648] improve the brtloader writer coverage
git-svn-id: file:///svn/toku/tokudb@20538 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Bradley C. Kuszmaul
59a8f06e93 The brtloader is now at 95.03% lines covered and 84.64% of branches taken. That's pretty good. We can work to remove a few more, but it's diminishing returns from here. [t:2647] Refs #2647.
git-svn-id: file:///svn/toku/tokudb@20537 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Bradley C. Kuszmaul
c4405b714c Make the ...dbufio tests better. Refs #2633. [t:2633]
git-svn-id: file:///svn/toku/tokudb@20534 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
1bba1e63b2 closes[t:2644] combine the brtloader panic and error callback state
git-svn-id: file:///svn/toku/tokudb@20526 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
418b60fbd7 refs[t:2643] compile the brtloader tests
git-svn-id: file:///svn/toku/tokudb@20489 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
af9d10bcd2 refs[t:2639] execute error return from kv_malloc
git-svn-id: file:///svn/toku/tokudb@20467 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
6c54d4ae28 closes[t:2639] recover from translation table realloc failures in the allocate block function of the brtloader. merge -r 20450:head from tokudb.2639
git-svn-id: file:///svn/toku/tokudb@20466 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
227abb8536 Fix #2533. (Avoid runts and don't make maximal fanout in loader). close[t:2533]
git-svn-id: file:///svn/toku/tokudb@20459 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
a6e22aadac Merge in the 2623 branch to main. Closes #2623. close[t:2623]
{{{
svn merge -r 20428:20432 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20433 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
5e03a29540 refs[t:2603] run some additional extractor tests
git-svn-id: file:///svn/toku/tokudb@20431 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
47c5441a8d Merge the basic dbufio working test in. It doesn't test destination files, and the failure cases aren't running yet. But the ...dbufio() code looks fairly well covered anyway. [t:2623] Refs #2623.
{{{
svn merge -r 20423:20425 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20426 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
48b56ff14f refs[t:2603] inject errors from the poll callback into the extractor
git-svn-id: file:///svn/toku/tokudb@20424 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Rich Prohaska
2651b61fd3 refs[t:2591] a better brtloader open test
git-svn-id: file:///svn/toku/tokudb@20418 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
661106bce3 Merge some changes back to main line (Refs #2623, #2631). [t:2623] [t:2631]
{{{
svn merge -r 20381:20412 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20413 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
d009c784dc closes[t:2626] fix the brtloader add_row realloc error recovery
git-svn-id: file:///svn/toku/tokudb@20385 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
5e75d244c1 Merge #2623 changes onto main line. The new test fails, but I need to get a merge from main again. Refs #2623. [t:2623]
{{{
svn merge -r 20348:20378 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20379 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
001f15c2f5 closes[t:2622] when realloc fails in the brtloader writer, mark the dbuf in error and check the error later
git-svn-id: file:///svn/toku/tokudb@20350 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
460ef9b4ca scale back the extractor error tests refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20321 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
abac9d89b3 refs[t:2603] run the brtloader tests in their own directories
git-svn-id: file:///svn/toku/tokudb@20310 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
432e750814 refs[t:2613] get the brtloader tests working with the new memory accounting
git-svn-id: file:///svn/toku/tokudb@20307 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
9e741558e2 test error from generate key callback refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20305 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Bradley C. Kuszmaul
e3d629fb2e Merge in memory management for the extractor thread. Refs #2613. [t:2613].
{{{
svn merge -r 20278:20301 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20302 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
d517a39ce5 recover from a big malloc failure in the middle of a rowset sort. merge -c 20283. refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20284 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
fbc02dfbec fix some mem leaks in the brtloader writer. merge -c 20280 to tokudb. refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20282 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
1abb93de95 test large malloc failures in the brtloader writer. merge from tokudb.2603.extractor.tests. refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20276 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00