Commit graph

73 commits

Author SHA1 Message Date
Yoni Fogel
7eb67d825b Windows port: ENODATA->TOKUDB_NO_DATA, disable unreachable code warning, disabled trace_mem for windows
git-svn-id: file:///svn/toku/tokudb@20400 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
127f1f927b refs[t:2626] fix some void* type conversions
git-svn-id: file:///svn/toku/tokudb@20389 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
d009c784dc closes[t:2626] fix the brtloader add_row realloc error recovery
git-svn-id: file:///svn/toku/tokudb@20385 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
5e75d244c1 Merge #2623 changes onto main line. The new test fails, but I need to get a merge from main again. Refs #2623. [t:2623]
{{{
svn merge -r 20348:20378 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20379 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
6f5c9f2fd4 Use {{{size_factor}}} to size rowsets. Fixes #2628. close[t:2628]
git-svn-id: file:///svn/toku/tokudb@20375 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
2108cc42c6 Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20364 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
8db89b8438 closes[t:2625] move the new assert macros to the toku_assert.h
git-svn-id: file:///svn/toku/tokudb@20353 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
001f15c2f5 closes[t:2622] when realloc fails in the brtloader writer, mark the dbuf in error and check the error later
git-svn-id: file:///svn/toku/tokudb@20350 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
d6dc7aad45 Change a MALLOC to an XMALLOC. Refs #2623. [t:2623]
git-svn-id: file:///svn/toku/tokudb@20347 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Bradley C. Kuszmaul
0b177d642b Merge the memory management into the main line. Refs #2613. [t:2613].
{{{
svn merge -r 20329:20340 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20341 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Rich Prohaska
7b852a3f96 remove an invariant to make the code work
git-svn-id: file:///svn/toku/tokudb@20337 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
c9d3fb0b97 more lazy_assert
git-svn-id: file:///svn/toku/tokudb@20336 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
d995208a1c remove lazy_assert in loader_do_put
git-svn-id: file:///svn/toku/tokudb@20335 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
366941f3cd fix leak with primary_rowset [t:2591]
git-svn-id: file:///svn/toku/tokudb@20334 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
b611db04c2 cleanup error handling in toku_brt_loader_open, toku_loader_open
git-svn-id: file:///svn/toku/tokudb@20331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
27cbb4c70a closes[t:2620] cleanup the lazy assert in the brtloader's close and abort functions
git-svn-id: file:///svn/toku/tokudb@20330 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
396aff4100 closes[t:2616] fix the file open fail path in the loader_do_i function
git-svn-id: file:///svn/toku/tokudb@20308 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
a59cb89830 change some lazy_asserts refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20306 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
9e741558e2 test error from generate key callback refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20305 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Bradley C. Kuszmaul
e3d629fb2e Merge in memory management for the extractor thread. Refs #2613. [t:2613].
{{{
svn merge -r 20278:20301 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20302 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
11897a5141 closes[t:2588] fix the loader create close test
git-svn-id: file:///svn/toku/tokudb@20298 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Dave Wells
f0b095543d assert labeling 3
git-svn-id: file:///svn/toku/tokudb@20296 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Dave Wells
e20411ddb1 assert labeling 2
git-svn-id: file:///svn/toku/tokudb@20289 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Dave Wells
23e3cd6b5f assert labeling
git-svn-id: file:///svn/toku/tokudb@20286 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
d517a39ce5 recover from a big malloc failure in the middle of a rowset sort. merge -c 20283. refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20284 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
fbc02dfbec fix some mem leaks in the brtloader writer. merge -c 20280 to tokudb. refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20282 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Dave Wells
829ba165fd new assert macros
git-svn-id: file:///svn/toku/tokudb@20279 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
1abb93de95 test large malloc failures in the brtloader writer. merge from tokudb.2603.extractor.tests. refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20276 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
0108e1d27a start testing big malloc failures in the brtloader extractor and writer refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20260 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
ecea4fe712 merge tokudb.2603 to tokudb refs[t:2603]
git-svn-id: file:///svn/toku/tokudb@20244 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Bradley C. Kuszmaul
c20a422a93 Reserve memory. Refs #2613. [t:2613].
git-svn-id: file:///svn/toku/tokudb@20241 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Bradley C. Kuszmaul
9dfcef61c1 Merge the 2571 double-buffering stuff over and delete the 2571 branch. Double buffering of the file merge is complete. Fixes #2571. close[t:2571]
{{{
svn merge -r 20199:20206 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20208 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
9c891e5501 fix some error paths in the brtloader extractor.
fix an uninitialized var problem in the ydbloader.
refs[t:2597] refs[t:2598]
merge -r 20145:20165 tokudb.2597 to tokudb



git-svn-id: file:///svn/toku/tokudb@20166 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
52b5bafc07 merge 2578b to main refs[t:2578]
git-svn-id: file:///svn/toku/tokudb@20143 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
1dcdd9e63b [t:2499] Allow test program to set loader's scale_factor. Improve loader-cleanup-test to exercise with different combinations of size_factor, ENOSPC from toku_os_write(), and abort_on_poll. Command line options are used to exercise known bugs.
git-svn-id: file:///svn/toku/tokudb@20109 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
0827928d76 Merge in the changes that make the reads not use fidx locks too. Refs #2571. [t:2571]
{{{
svn merge -r 20067:20072 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20073 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
28ba5d6e76 Merge and retstart 2571. Refs #2571. [t:2571]
{{{
svn merge -r 20030:20036 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20066 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
76e85d0b61 fix the loader-dup-test mem leaks closes[t:2594] #2594
git-svn-id: file:///svn/toku/tokudb@20064 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
699037a889 Set the queue to depth=2 for the extractor_thread. Refs #2571. [t:2571]
git-svn-id: file:///svn/toku/tokudb@20032 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Bradley C. Kuszmaul
47f2199d9d Merge in
* bradley's benchmarking changes, and
 * the big speedup in the sort-and-write-rows (slowness caused the lock in fidx2file).
Refs #2571. [t:2571]
{{{
svn merge -r 19909:20027 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20029 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
c8e4582e59 merge -r 20020:20023 tokudb.2578 to main. build the brtloader with cilk++. refs[t:2578]
git-svn-id: file:///svn/toku/tokudb@20024 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
84519c2e5b get the loader-cleanup-test to run to completion refs[t:2578]
git-svn-id: file:///svn/toku/tokudb@20019 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
bb7d5414f2 shutdown the extractor in loader abort. closes[t:2586] #2586
git-svn-id: file:///svn/toku/tokudb@20013 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Yoni Fogel
d91de8331a [t:2499] Windows port.
*pthread* -> *toku_pthread*
Printf format issues
Including toku_portability and toku_os before everything else

git-svn-id: file:///svn/toku/tokudb@19938 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
6eae60da8a remove the 'deq' printf refs[t:2571]
git-svn-id: file:///svn/toku/tokudb@19932 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Rich Prohaska
486decfd7f put the loader error callback in its own source file so we can reduce dependancies on cilk refs[t:2576]
git-svn-id: file:///svn/toku/tokudb@19931 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Rich Prohaska
0612028bcf fix the loader error callback and the pqueue-test closes[t:2576]
git-svn-id: file:///svn/toku/tokudb@19913 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
9cab06f7fb Merge the instrumentation from 2499d+2571 onto the main line (since 2499d has been merged.) Refs #2571. [t:2571].
{{{
svn merge -r 19857:19905 https://svn.tokutek.com/tokudb/toku/tokudb.2499d+2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@19906 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
450d693d2a Remove extra cilk context. Fixes #2575. close[t:2575].
git-svn-id: file:///svn/toku/tokudb@19905 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
28cc5d1ed9 Merge the changes from 2499d onto the main line. Fixes #2499. close[t:2499].
{{{
svn merge -r 19523:19895 https://svn.tokutek.com/tokudb/toku/tokudb.2499d
}}}
.


git-svn-id: file:///svn/toku/tokudb@19902 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00