Commit graph

138 commits

Author SHA1 Message Date
Rich Prohaska
28877528fb #3678 use clean or mvcc leaf entries when appropriate refs[t:3678]
git-svn-id: file:///svn/toku/tokudb@32718 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:42 -04:00
Bradley C. Kuszmaul
2379d061e9 [t:3564] Merge in barry's stuff to the main line. {{{svn merge -r32637:32693 ../tokudb.3564b+3312}}} Refs #3564.
git-svn-id: file:///svn/toku/tokudb@32694 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:42 -04:00
Rich Prohaska
bb69802bed #3588 merge tokudb.3588 to main refs[t:3588]
git-svn-id: file:///svn/toku/tokudb@32669 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:42 -04:00
Zardosht Kasheff
cd757a485d [t:3641], merge refactoring of BRTNODE to main from tokudb.3627
git-svn-id: file:///svn/toku/tokudb@32481 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:41 -04:00
Bradley C. Kuszmaul
885c83300b [t:3604] Merge everything from the tokudb.3312c+3524+msn branch. Refs #3604.
git-svn-id: file:///svn/toku/tokudb@32294 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:40 -04:00
Bradley C. Kuszmaul
866597e6d1 [t:3586] [t:3604] Merge icc onto main. Refs #3586, #3604.
git-svn-id: file:///svn/toku/tokudb@32232 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:40 -04:00
Rich Prohaska
92244247d3 #3462 fix the brtloader leaf builder refs[t:3462]
git-svn-id: file:///svn/toku/tokudb@30309 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:39 -04:00
Rich Prohaska
a2d62ea401 #3458 use the brt default node size in the brtloader refs[t:3458]
git-svn-id: file:///svn/toku/tokudb@30269 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:39 -04:00
Bradley C. Kuszmaul
7e7e93103c #3316 #3318 #3339
* #3316 quicklz is now part of the brt layer (but it won't be enabled for compression in 5.0.2, #3339)
 * #3318 (add valgrind suppresseions)
{{{
svn merge -r28917:29103 https://svn.tokutek.com/tokudb/toku/tokudb.3316
}}}
.
[t:3316] [t:3318] [t:3339]


git-svn-id: file:///svn/toku/tokudb@29107 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:37 -04:00
Bradley C. Kuszmaul
80089f63a4 #3099 Merge 3099 onto the main line.
{{{
svn merge -r28775:29048 https://svn.tokutek.com/tokudb/toku/tokudb.3099
}}}


git-svn-id: file:///svn/toku/tokudb@29055 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:37 -04:00
Barry Perlman
036c8d6b54 [t:3236] [t:3238] Merge tokudb.3236+3238 to main.
git-svn-id: file:///svn/toku/tokudb@28070 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:36 -04:00
Rich Prohaska
cda81a2103 #3202 fix =- typo refs[t:3202]
git-svn-id: file:///svn/toku/tokudb@27499 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:35 -04:00
Rich Prohaska
5297be0f96 #3164 fix changeset 27062 refs[t:3164]
git-svn-id: file:///svn/toku/tokudb@27073 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:34 -04:00
Rich Prohaska
ecb82d778e #3164 fix the brtloader pivot key problem refs[t:3164]
git-svn-id: file:///svn/toku/tokudb@27062 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:34 -04:00
Rich Prohaska
42921bb831 #3163 use the nodesize configured in a db in the brtloader refs[t:3163]
git-svn-id: file:///svn/toku/tokudb@26909 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:34 -04:00
Barry Perlman
c68e4f968a [t:3102] Change brtloader_fi_close() to ignore files that are already closed when called from brtloader_fi_close_all().
git-svn-id: file:///svn/toku/tokudb@26361 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:33 -04:00
Barry Perlman
2f4f4976be [t:3102] Restore brtloader_fi_close_all() to brtloader-internal.h (needed by test programs). Also clarify use of return values in brtloader_fi_close().
git-svn-id: file:///svn/toku/tokudb@26360 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:33 -04:00
Barry Perlman
de61796b6c [t:3102] Propogate error return up call stack when fclose() fails. Make brtloader_fi_close_all() static to brtloader.c and delete its signature from brtloader-internal.h.
git-svn-id: file:///svn/toku/tokudb@26318 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:33 -04:00
Rich Prohaska
43e5039e58 #3011 remove *extra from *multiple functions and callbacks refs[t:3011]
git-svn-id: file:///svn/toku/tokudb@25627 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:30 -04:00
Barry Perlman
548d03d70e [t:2892] Merge upgrade logic to main. Merge command was svn merge --accept=postpone -r25293:HEAD ../tokudb.main+2892 .
git-svn-id: file:///svn/toku/tokudb@25303 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:25 -04:00
Rich Prohaska
24d915f259 #2966 use assert_zero in some of the newbrt files refs[t:2966]
git-svn-id: file:///svn/toku/tokudb@24717 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:23 -04:00
Rich Prohaska
5c0dce7cbb #2513 merge the cached threadpool to main refs[t:2513]
git-svn-id: file:///svn/toku/tokudb@24627 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:23 -04:00
Zardosht Kasheff
cc91ea8ca4 [t:2811], merge to main \!\!\!
git-svn-id: file:///svn/toku/tokudb@23495 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:22 -04:00
Rich Prohaska
59a83a0814 #2868 refs[t:2868] comments from code review
git-svn-id: file:///svn/toku/tokudb@22875 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:21 -04:00
Rich Prohaska
ab1d8172bd #2868 refs[t:2868] fix the presort speedup in the loader
git-svn-id: file:///svn/toku/tokudb@22868 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:21 -04:00
Rich Prohaska
d9985ad0cf #2840 refs[t:2840] add debug prints to the loader to help fix the dbufio read row problem, also remove the assert on this error path
git-svn-id: file:///svn/toku/tokudb@22673 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:20 -04:00
Rich Prohaska
44d24685b8 #2833 refs[t:2833] fix the brtloader to use less memory during key extraction and merge sorting
git-svn-id: file:///svn/toku/tokudb@22581 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:20 -04:00
Rich Prohaska
7a3f6089e2 #2821 refs[t:2821] fixup the loader tests
git-svn-id: file:///svn/toku/tokudb@22416 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:20 -04:00
Rich Prohaska
6f67fad660 #2821 #2829 refs[t:2821] refs[t:2829] fix the loader's merger vmsize and remove an fsync bubble
git-svn-id: file:///svn/toku/tokudb@22413 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:20 -04:00
Rich Prohaska
663acdae84 #2654 refs[t:2654] merge loader persort speedup to main tokudb
git-svn-id: file:///svn/toku/tokudb@22354 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:20 -04:00
Bradley C. Kuszmaul
07f7970808 Merge 2577c onto main line. Refs #2755c. [t:2755c]
{{{
svn merge -r22020:22070 https://svn.tokutek.com/tokudb/toku/tokudb.2755c
}}}
.


git-svn-id: file:///svn/toku/tokudb@22160 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:20 -04:00
Bradley C. Kuszmaul
c49d7a8c0c The first loader gets 1/2 of cachetable memory.
* That's 2/3 of 3/4, which is calculated as 1/4 is unreservable.  The loader asks for 2/3 of what's reservable, getting one half.
The second loader gets 1/6 of cachetable memory.
 * That's 2/3 of 1/4: The loader asks for 2/3 of what's reservable.  1/2 has been reserved.  1/4 is unreservable.  1/4 is reservable.  2/3 of 1/4 is 1/6.
The third loader gets 1/18 of cachetable memory. 
 * That's 2/3 or 1/12.   (1/4 is unreservable. 1/2 was reserved by the first loader.  1/6 by the second.  That leaves 1/12)
Generally the next one gets 1/3 of what the previous one got.

Refs #2759. [t:2759]


git-svn-id: file:///svn/toku/tokudb@21564 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Rich Prohaska
c37567705f #2738 refs[t:2738] merge dbufio bug fix to main
git-svn-id: file:///svn/toku/tokudb@21260 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Rich Prohaska
1a65ba1d80 #2723 refs[t:2723] fix the slow control-c reaction in the brtloader's write to brt function
git-svn-id: file:///svn/toku/tokudb@21103 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Bradley C. Kuszmaul
dfdaee296d Candidate fix for #2730 (the race in the loader and the io-thread where they glare at each other). Refs #2730. [t:2730]
git-svn-id: file:///svn/toku/tokudb@21082 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Rich Prohaska
6b09687c83 #2718 refs[t:2718] free the big buffer attached to the temp files when they are closed
git-svn-id: file:///svn/toku/tokudb@21045 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Rich Prohaska
8a95226d84 #2717 refs[t:2717] fix the brtloader subtree estimates
git-svn-id: file:///svn/toku/tokudb@21040 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:19 -04:00
Yoni Fogel
c9b2b066dd [t:2561] Merge tokudb.2561b to main. Disabled auto-upgrade and auto-upgrade tests
git-svn-id: file:///svn/toku/tokudb@20778 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:17 -04:00
Rich Prohaska
16cbf13ca2 closes[t:2445] fsync dirs after file creation
git-svn-id: file:///svn/toku/tokudb@20735 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Dave Wells
85bfc804f0 remove cilk_spawn that is within cilk_for loop
git-svn-id: file:///svn/toku/tokudb@20684 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
e6a7af95a3 refs[t:2654] fix a NULL pointer deref found by valgrind brtloader-test-open
git-svn-id: file:///svn/toku/tokudb@20672 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Rich Prohaska
d86d1e5361 closes[t:2659] fix the data race in the finish_primary_rows_internal function. found by cilkscreen ./brtloader-test-extractor -s
git-svn-id: file:///svn/toku/tokudb@20657 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Bradley C. Kuszmaul
54490d3c06 Simplify structures to keep track of whether data in an index is sorted. Refs #2654. [t:2654].
git-svn-id: file:///svn/toku/tokudb@20644 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:16 -04:00
Bradley C. Kuszmaul
57fb43c9c5 Add structures to keep track of whether data in an index is sorted. We aren't maintaining it properly yet, but everything should work since we aren't using it. Refs #2654. [t:2654].
git-svn-id: file:///svn/toku/tokudb@20640 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
9e02290285 closes[t:2642] fix a mem leak in the brtloader found with the cleanup test
git-svn-id: file:///svn/toku/tokudb@20627 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
5518290137 closes[t:2591] fix the brtloader destroy to avoid the mutex when not yet initialized
git-svn-id: file:///svn/toku/tokudb@20608 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
42fdb90153 refs[t:2591] mutex destroy called on an uninitalized mutex
git-svn-id: file:///svn/toku/tokudb@20557 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Bradley C. Kuszmaul
d14326cc0e brtloader.c is now 96.61% of lines and 87.13% of branches. [t:2647] Refs #2647.
git-svn-id: file:///svn/toku/tokudb@20539 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Rich Prohaska
543dd95cdb closes[t:2648] improve the brtloader writer coverage
git-svn-id: file:///svn/toku/tokudb@20538 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00
Bradley C. Kuszmaul
59a8f06e93 The brtloader is now at 95.03% lines covered and 84.64% of branches taken. That's pretty good. We can work to remove a few more, but it's diminishing returns from here. [t:2647] Refs #2647.
git-svn-id: file:///svn/toku/tokudb@20537 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:15 -04:00