Barry Perlman
1dcdd9e63b
[t:2499] Allow test program to set loader's scale_factor. Improve loader-cleanup-test to exercise with different combinations of size_factor, ENOSPC from toku_os_write(), and abort_on_poll. Command line options are used to exercise known bugs.
...
git-svn-id: file:///svn/toku/tokudb@20109 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
0827928d76
Merge in the changes that make the reads not use fidx locks too. Refs #2571 . [t:2571]
...
{{{
svn merge -r 20067:20072 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.
git-svn-id: file:///svn/toku/tokudb@20073 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
28ba5d6e76
Merge and retstart 2571. Refs #2571 . [t:2571]
...
{{{
svn merge -r 20030:20036 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.
git-svn-id: file:///svn/toku/tokudb@20066 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
76e85d0b61
fix the loader-dup-test mem leaks closes[t:2594] #2594
...
git-svn-id: file:///svn/toku/tokudb@20064 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
699037a889
Set the queue to depth=2 for the extractor_thread. Refs #2571 . [t:2571]
...
git-svn-id: file:///svn/toku/tokudb@20032 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Bradley C. Kuszmaul
47f2199d9d
Merge in
...
* bradley's benchmarking changes, and
* the big speedup in the sort-and-write-rows (slowness caused the lock in fidx2file).
Refs #2571 . [t:2571]
{{{
svn merge -r 19909:20027 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.
git-svn-id: file:///svn/toku/tokudb@20029 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
c8e4582e59
merge -r 20020:20023 tokudb.2578 to main. build the brtloader with cilk++. refs[t:2578]
...
git-svn-id: file:///svn/toku/tokudb@20024 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
84519c2e5b
get the loader-cleanup-test to run to completion refs[t:2578]
...
git-svn-id: file:///svn/toku/tokudb@20019 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
bb7d5414f2
shutdown the extractor in loader abort. closes[t:2586] #2586
...
git-svn-id: file:///svn/toku/tokudb@20013 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Yoni Fogel
d91de8331a
[t:2499] Windows port.
...
*pthread* -> *toku_pthread*
Printf format issues
Including toku_portability and toku_os before everything else
git-svn-id: file:///svn/toku/tokudb@19938 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
6eae60da8a
remove the 'deq' printf refs[t:2571]
...
git-svn-id: file:///svn/toku/tokudb@19932 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Rich Prohaska
486decfd7f
put the loader error callback in its own source file so we can reduce dependancies on cilk refs[t:2576]
...
git-svn-id: file:///svn/toku/tokudb@19931 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Rich Prohaska
0612028bcf
fix the loader error callback and the pqueue-test closes[t:2576]
...
git-svn-id: file:///svn/toku/tokudb@19913 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
9cab06f7fb
Merge the instrumentation from 2499d+2571 onto the main line (since 2499d has been merged.) Refs #2571 . [t:2571].
...
{{{
svn merge -r 19857:19905 https://svn.tokutek.com/tokudb/toku/tokudb.2499d+2571
}}}
.
git-svn-id: file:///svn/toku/tokudb@19906 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
450d693d2a
Remove extra cilk context. Fixes #2575 . close[t:2575].
...
git-svn-id: file:///svn/toku/tokudb@19905 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
28cc5d1ed9
Merge the changes from 2499d onto the main line. Fixes #2499 . close[t:2499].
...
{{{
svn merge -r 19523:19895 https://svn.tokutek.com/tokudb/toku/tokudb.2499d
}}}
.
git-svn-id: file:///svn/toku/tokudb@19902 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
939fd20e37
[t:2499] Delete extraneous ggg from first line.
...
git-svn-id: file:///svn/toku/tokudb@19834 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Barry Perlman
0fc32fb911
[t:2449] Export test function (override loader's fwrite) via env. Fix bug in brtloader's handling of error from overridden fwrite().
...
Fix bug in ydb's handling of ephemeral loader when unable to write to disk.
Improve loader-cleanup-test.
git-svn-id: file:///svn/toku/tokudb@19825 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Dave Wells
fa39625528
change assert to use == instead of = as per icc error
...
git-svn-id: file:///svn/toku/tokudb@19819 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Yoni Fogel
36895dcfce
Revert "[t:2555] Prevent toku_free(NULL) in regular (not LOADER_USE_PUTS) loader."
...
This reverts commit 755bb963ba9c47fcf2c90da195a15544be100dc6.
This reverts svn commit r19634
toku_free(NULL) is safe. No reason to prevent it.
git-svn-id: file:///svn/toku/tokudb@19685 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Yoni Fogel
9542e88667
[t:2555] Prevent toku_free(NULL) in regular (not LOADER_USE_PUTS) loader.
...
If generate_row_for_put sets flags to 0, skey/sval.data are NULL and we unconditionally free them.
git-svn-id: file:///svn/toku/tokudb@19634 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:07 -04:00
Yoni Fogel
060e15681e
[t:2538] Fix memset usage
...
git-svn-id: file:///svn/toku/tokudb@19553 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:07 -04:00
Rich Prohaska
3f35cb893b
change block format to support leaf partitions closes[t:2351]
...
git-svn-id: file:///svn/toku/tokudb@19500 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:07 -04:00
Yoni Fogel
65f91e8663
Windows port minor changes
...
git-svn-id: file:///svn/toku/tokudb@19227 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:06 -04:00
Yoni Fogel
667e46d60c
closes[t:2483] Fix recovery bug in loader.
...
checkpoint_lsn written to headers made by loader are now the fsynced 'load' log entry's lsn instead of MAX_UINT64
git-svn-id: file:///svn/toku/tokudb@19136 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:05 -04:00
Rich Prohaska
48414ab76b
create multiple sub blocks per leaf in the loader. merge -r 18961:head ptq branch to main refs[t:2351]
...
git-svn-id: file:///svn/toku/tokudb@18992 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:05 -04:00
Rich Prohaska
c1b0e91b95
new block format with a header checksum and sub block checksums. merge -c 18916:head from 2351.ptq to main refs[t:2351]
...
git-svn-id: file:///svn/toku/tokudb@18961 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:05 -04:00
Bradley C. Kuszmaul
a3a44f8e57
Fixes #2487 . close[t:2487]. One of the problems was a buffer overflow problem because the header hadn't been extended.
...
git-svn-id: file:///svn/toku/tokudb@18900 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:04 -04:00
Bradley C. Kuszmaul
e97dc367c0
Set compresison_level to 5
...
git-svn-id: file:///svn/toku/tokudb@18836 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:04 -04:00
Bradley C. Kuszmaul
3bfdfa2921
Get rid of the gratuitous IDX files.
...
git-svn-id: file:///svn/toku/tokudb@18835 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:04 -04:00
Dave Wells
c9e32bbe03
add toku_brt_loader_abort, fix up error handling in toku_loader_*
...
git-svn-id: file:///svn/toku/tokudb@18804 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:04 -04:00
Bradley C. Kuszmaul
3919e85519
Merge the finer-grained progress report for the loader. Refs #2462 . [t:2462]
...
{{{
svn merge -r 18775:18791 https://svn.tokutek.com/tokudb/toku/tokudb.2462
}}}
.
git-svn-id: file:///svn/toku/tokudb@18792 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:04 -04:00
Bradley C. Kuszmaul
eb9ad2a896
Merge in the 2462 branch that has a rudimentary implementation of polling. Refs #2462 . [t:2462]
...
{{{
svn merge -c 18768 https://svn.tokutek.com/tokudb/toku/tokudb.2462
}}}
.
git-svn-id: file:///svn/toku/tokudb@18771 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:03 -04:00
Yoni Fogel
b33a8c1830
[t:2216] windows port of #2216 merge (remove statement expressions)
...
git-svn-id: file:///svn/toku/tokudb@18753 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:03 -04:00
Bradley C. Kuszmaul
788b96e407
Merge 2216b onto main line. Refs #2216 . [t:2216]
...
{{{
svn merge -r18206:18672 https://svn.tokutek.com/tokudb/toku/tokudb.2216b
svn merge -r 18738:18746 https://svn.tokutek.com/tokudb/toku/tokudb.2216b
}}}
.
git-svn-id: file:///svn/toku/tokudb@18749 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:03 -04:00
Yoni Fogel
b3f5306b41
[t:2249] Merge #2249 to main.
...
'load' entry and rollback/recovery,
single filename for brtloader and rest of brt
test-fsync
git-svn-id: file:///svn/toku/tokudb@18617 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:03 -04:00
Bradley C. Kuszmaul
099c51de5b
Fix layout version in brtloader header. Refs #2216 . [t:2216].
...
git-svn-id: file:///svn/toku/tokudb@18276 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:02 -04:00
Bradley C. Kuszmaul
178820a688
Merge changes from 2216a to main (except possibly for the windows subdirectory, which Yoni will do separately).
...
2216a is no longer valid.
Refs #2216 . [t:2216].
{{{
svn merge -r17301:18179 https://svn.tokutek.com/tokudb/toku/tokudb.2216a/src/tests
}}}
git-svn-id: file:///svn/toku/tokudb@18181 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:01 -04:00
Bradley C. Kuszmaul
56d2c6fdf8
Merge 2216 onto main (from 16706-17293) as
...
{{{
svn merge -r16706:17293 https://svn.tokutek.com/tokudb/toku/tokudb.2216
}}}
Refs #2216 . [t:2216]
git-svn-id: file:///svn/toku/tokudb@17299 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:59 -04:00