Commit graph

1569 commits

Author SHA1 Message Date
Barry Perlman
8b7ecdf800 [t:2499] loader-cleanup-test: Improve error reporting
git-svn-id: file:///svn/toku/tokudb@20429 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
53cb67c979 Merge some tests of error paths back to main line. Refs #2623. [t:2623]
{{{
svn merge -r 20415:20420 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20421 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Dave Wells
02e2cc1595 reverse polarity of -M option - now default behavior is use half of memory for cachesize
git-svn-id: file:///svn/toku/tokudb@20417 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:14 -04:00
Bradley C. Kuszmaul
661106bce3 Merge some changes back to main line (Refs #2623, #2631). [t:2623] [t:2631]
{{{
svn merge -r 20381:20412 https://svn.tokutek.com/tokudb/toku/tokudb.2623
}}}
.


git-svn-id: file:///svn/toku/tokudb@20413 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
7e9e6af90a [t:2499] loader-cleanup-test: finer control over verbosity.
git-svn-id: file:///svn/toku/tokudb@20391 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
3da94d6415 [t:2499] Use correct LOADER_USE_PUTS flag. Add engine status output when verbose>=2.
git-svn-id: file:///svn/toku/tokudb@20374 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
2108cc42c6 Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20364 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
9ccf76833d Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20360 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
939a4ea0c5 [t:2499] Add ability to control size_factor and count temp files.
git-svn-id: file:///svn/toku/tokudb@20359 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
b611db04c2 cleanup error handling in toku_brt_loader_open, toku_loader_open
git-svn-id: file:///svn/toku/tokudb@20331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Bradley C. Kuszmaul
e3d629fb2e Merge in memory management for the extractor thread. Refs #2613. [t:2613].
{{{
svn merge -r 20278:20301 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20302 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
f85b67f78a [t:2499] Loader temp files go in data dir, not env home dir (unless there is no data dir).
git-svn-id: file:///svn/toku/tokudb@20299 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
11897a5141 closes[t:2588] fix the loader create close test
git-svn-id: file:///svn/toku/tokudb@20298 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
6603cf2166 [t:2499] Fix toku_os_pwrite() signature (returns ssize_t, not int).Add pwrite testing to loader-cleanup-test.
git-svn-id: file:///svn/toku/tokudb@20292 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Dave Wells
23e3cd6b5f assert labeling
git-svn-id: file:///svn/toku/tokudb@20286 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
eaa5564cbd [t:2499] Induce ENOSPC return from toku_os_write().
git-svn-id: file:///svn/toku/tokudb@20221 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
061de97e14 Closes [t:2612] Make separate override functions for use by toku_os_write(), toku_os_full_write(), toku_os_pwrite(), and toku_os_full_pwrite(). Modify src/tests/diskfull.c to use new override functions.
git-svn-id: file:///svn/toku/tokudb@20220 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
9dfcef61c1 Merge the 2571 double-buffering stuff over and delete the 2571 branch. Double buffering of the file merge is complete. Fixes #2571. close[t:2571]
{{{
svn merge -r 20199:20206 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20208 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
cd86a31e33 [t:2499] Replace multiple free_loader() calls for each error condition in toku_loader_create_loader() with single call on exit.
git-svn-id: file:///svn/toku/tokudb@20207 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Dave Wells
aa244950db remove lsn checking now that checkpoints are enabled
git-svn-id: file:///svn/toku/tokudb@20200 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
cd7927ea01 [t:2499] loader-cleanup-test Remove -a and -e options, make default.
git-svn-id: file:///svn/toku/tokudb@20170 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
9c891e5501 fix some error paths in the brtloader extractor.
fix an uninitialized var problem in the ydbloader.
refs[t:2597] refs[t:2598]
merge -r 20145:20165 tokudb.2597 to tokudb



git-svn-id: file:///svn/toku/tokudb@20166 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
e0fd8d4fad [t:2499] Bugfix in loader-cleanup-test, free inames.
git-svn-id: file:///svn/toku/tokudb@20164 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Dave Wells
507e490c6e change temp prefix to tokuld, closes #2536 [t:2536]
git-svn-id: file:///svn/toku/tokudb@20149 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
d2decf3fc8 [t:2504] Bug fix, do not attempt to update accountability counters when toku_lt_acquire_range_read_lock() or toku_lt_acquire_range_write_lock() is called with NULL tree.
git-svn-id: file:///svn/toku/tokudb@20142 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
9ffaaf4525 fix the library globals error filter
git-svn-id: file:///svn/toku/tokudb@20136 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
1dcdd9e63b [t:2499] Allow test program to set loader's scale_factor. Improve loader-cleanup-test to exercise with different combinations of size_factor, ENOSPC from toku_os_write(), and abort_on_poll. Command line options are used to exercise known bugs.
git-svn-id: file:///svn/toku/tokudb@20109 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
a02e4d8786 [t:2499] Fix previous fix. Correctly insert write errors at last few writes.
git-svn-id: file:///svn/toku/tokudb@20096 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
ae050bf956 [t:2504] Improve engine status.
git-svn-id: file:///svn/toku/tokudb@20077 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
76e85d0b61 fix the loader-dup-test mem leaks closes[t:2594] #2594
git-svn-id: file:///svn/toku/tokudb@20064 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
350ae69211 start debugging the loader-dup-test leaks refs[t:2594]
git-svn-id: file:///svn/toku/tokudb@20063 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
3284a89f45 add the -p option to the loader-create-abort test refs[t:2588] #2588
git-svn-id: file:///svn/toku/tokudb@20061 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
d8e31c8fcc add the -p option to the loader-create-close test refs[t:2588] #2588
git-svn-id: file:///svn/toku/tokudb@20060 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
9a7a5ddef1 fix the loader cleanup test refs[t:2578]
git-svn-id: file:///svn/toku/tokudb@20035 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Bradley C. Kuszmaul
47f2199d9d Merge in
* bradley's benchmarking changes, and
 * the big speedup in the sort-and-write-rows (slowness caused the lock in fidx2file).
Refs #2571. [t:2571]
{{{
svn merge -r 19909:20027 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20029 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Barry Perlman
4b42c354b2 [t:2499] Induce ENOSPC errors in more places, some concentrated at beginning of load, some concentrated at end, some sprinkled in the middle."
git-svn-id: file:///svn/toku/tokudb@20025 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
8fe975666d loader create close hits an assert refs[t:2588] #2588
git-svn-id: file:///svn/toku/tokudb@19999 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
2146a8c2f7 add a simple test that exibits the loader->abort memory leaks refs[t:2586] #2586
git-svn-id: file:///svn/toku/tokudb@19997 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Yoni Fogel
034da281ee [t:2499]Added TOKUDB_CANCELED error code for use in canceling during poll function of loader.
git-svn-id: file:///svn/toku/tokudb@19950 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
5a3875a5ea cleanup old loader temp files when the env is opened closes[t:2550]
git-svn-id: file:///svn/toku/tokudb@19941 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Yoni Fogel
451f3e57fc [t:2499] Windows port.
ECANCELED does not exist in windows
made some functions static

git-svn-id: file:///svn/toku/tokudb@19939 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Barry Perlman
b9604f998e [t:2499] Set errno so that brtloader's bl_fwrite() function gets correct error return if the real fwrite() produces an error.
git-svn-id: file:///svn/toku/tokudb@19934 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
a9b7c490d7 [t:2499] Increase number of rows by 500X to induce creation of temp files.
git-svn-id: file:///svn/toku/tokudb@19930 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
a4eedcb492 Closes [t:2577] Fixed unitialized variable.
git-svn-id: file:///svn/toku/tokudb@19914 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
dd51442ed6 [t:2499] Verify loader-created iname files are deleted on recovery.
git-svn-id: file:///svn/toku/tokudb@19904 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
e534524e76 [t:2499] Comment changes only.
git-svn-id: file:///svn/toku/tokudb@19903 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
28cc5d1ed9 Merge the changes from 2499d onto the main line. Fixes #2499. close[t:2499].
{{{
svn merge -r 19523:19895 https://svn.tokutek.com/tokudb/toku/tokudb.2499d
}}}
.


git-svn-id: file:///svn/toku/tokudb@19902 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
82053e53d3 [t:2499] Verify no temp files before loader is run, print data_dir listing only if verbose
git-svn-id: file:///svn/toku/tokudb@19901 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Barry Perlman
84b65004df [t:2499] Add recover-loader-test
git-svn-id: file:///svn/toku/tokudb@19900 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Rich Prohaska
c37f8a21a6 fix the globals test refs[t:2449]
git-svn-id: file:///svn/toku/tokudb@19882 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00