Commit graph

1665 commits

Author SHA1 Message Date
Barry Perlman
7e9e6af90a [t:2499] loader-cleanup-test: finer control over verbosity.
git-svn-id: file:///svn/toku/tokudb@20391 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
3da94d6415 [t:2499] Use correct LOADER_USE_PUTS flag. Add engine status output when verbose>=2.
git-svn-id: file:///svn/toku/tokudb@20374 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
2108cc42c6 Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20364 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
9ccf76833d Closes [t:2624] Added toku_os_close() which retries on EINTR. Closes [t:2621] Added toku_os_open(), toku_os_fopen(), toku_os_fdopen(), toku_os_fclose() and related override hooks db_env_set_func_open(), db_env_set_func_fopen(), db_env_set_func_fdopen(), db_env_set_func_fclose(). Modified test program loader-cleanup-test to use new hooks to inject synthetic errors. Marked some error cases in brtloader.c. with ###.
git-svn-id: file:///svn/toku/tokudb@20360 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Barry Perlman
939a4ea0c5 [t:2499] Add ability to control size_factor and count temp files.
git-svn-id: file:///svn/toku/tokudb@20359 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:13 -04:00
Dave Wells
b611db04c2 cleanup error handling in toku_brt_loader_open, toku_loader_open
git-svn-id: file:///svn/toku/tokudb@20331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Bradley C. Kuszmaul
e3d629fb2e Merge in memory management for the extractor thread. Refs #2613. [t:2613].
{{{
svn merge -r 20278:20301 https://svn.tokutek.com/tokudb/toku/tokudb.2613
}}}
.


git-svn-id: file:///svn/toku/tokudb@20302 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
f85b67f78a [t:2499] Loader temp files go in data dir, not env home dir (unless there is no data dir).
git-svn-id: file:///svn/toku/tokudb@20299 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Rich Prohaska
11897a5141 closes[t:2588] fix the loader create close test
git-svn-id: file:///svn/toku/tokudb@20298 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
6603cf2166 [t:2499] Fix toku_os_pwrite() signature (returns ssize_t, not int).Add pwrite testing to loader-cleanup-test.
git-svn-id: file:///svn/toku/tokudb@20292 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Dave Wells
23e3cd6b5f assert labeling
git-svn-id: file:///svn/toku/tokudb@20286 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
eaa5564cbd [t:2499] Induce ENOSPC return from toku_os_write().
git-svn-id: file:///svn/toku/tokudb@20221 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:12 -04:00
Barry Perlman
061de97e14 Closes [t:2612] Make separate override functions for use by toku_os_write(), toku_os_full_write(), toku_os_pwrite(), and toku_os_full_pwrite(). Modify src/tests/diskfull.c to use new override functions.
git-svn-id: file:///svn/toku/tokudb@20220 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Bradley C. Kuszmaul
9dfcef61c1 Merge the 2571 double-buffering stuff over and delete the 2571 branch. Double buffering of the file merge is complete. Fixes #2571. close[t:2571]
{{{
svn merge -r 20199:20206 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20208 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
cd86a31e33 [t:2499] Replace multiple free_loader() calls for each error condition in toku_loader_create_loader() with single call on exit.
git-svn-id: file:///svn/toku/tokudb@20207 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Dave Wells
aa244950db remove lsn checking now that checkpoints are enabled
git-svn-id: file:///svn/toku/tokudb@20200 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
cd7927ea01 [t:2499] loader-cleanup-test Remove -a and -e options, make default.
git-svn-id: file:///svn/toku/tokudb@20170 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
9c891e5501 fix some error paths in the brtloader extractor.
fix an uninitialized var problem in the ydbloader.
refs[t:2597] refs[t:2598]
merge -r 20145:20165 tokudb.2597 to tokudb



git-svn-id: file:///svn/toku/tokudb@20166 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
e0fd8d4fad [t:2499] Bugfix in loader-cleanup-test, free inames.
git-svn-id: file:///svn/toku/tokudb@20164 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Dave Wells
507e490c6e change temp prefix to tokuld, closes #2536 [t:2536]
git-svn-id: file:///svn/toku/tokudb@20149 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
d2decf3fc8 [t:2504] Bug fix, do not attempt to update accountability counters when toku_lt_acquire_range_read_lock() or toku_lt_acquire_range_write_lock() is called with NULL tree.
git-svn-id: file:///svn/toku/tokudb@20142 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
9ffaaf4525 fix the library globals error filter
git-svn-id: file:///svn/toku/tokudb@20136 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
1dcdd9e63b [t:2499] Allow test program to set loader's scale_factor. Improve loader-cleanup-test to exercise with different combinations of size_factor, ENOSPC from toku_os_write(), and abort_on_poll. Command line options are used to exercise known bugs.
git-svn-id: file:///svn/toku/tokudb@20109 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
a02e4d8786 [t:2499] Fix previous fix. Correctly insert write errors at last few writes.
git-svn-id: file:///svn/toku/tokudb@20096 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Barry Perlman
ae050bf956 [t:2504] Improve engine status.
git-svn-id: file:///svn/toku/tokudb@20077 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
76e85d0b61 fix the loader-dup-test mem leaks closes[t:2594] #2594
git-svn-id: file:///svn/toku/tokudb@20064 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:11 -04:00
Rich Prohaska
350ae69211 start debugging the loader-dup-test leaks refs[t:2594]
git-svn-id: file:///svn/toku/tokudb@20063 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
3284a89f45 add the -p option to the loader-create-abort test refs[t:2588] #2588
git-svn-id: file:///svn/toku/tokudb@20061 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
d8e31c8fcc add the -p option to the loader-create-close test refs[t:2588] #2588
git-svn-id: file:///svn/toku/tokudb@20060 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
9a7a5ddef1 fix the loader cleanup test refs[t:2578]
git-svn-id: file:///svn/toku/tokudb@20035 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Bradley C. Kuszmaul
47f2199d9d Merge in
* bradley's benchmarking changes, and
 * the big speedup in the sort-and-write-rows (slowness caused the lock in fidx2file).
Refs #2571. [t:2571]
{{{
svn merge -r 19909:20027 https://svn.tokutek.com/tokudb/toku/tokudb.2571
}}}
.


git-svn-id: file:///svn/toku/tokudb@20029 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Barry Perlman
4b42c354b2 [t:2499] Induce ENOSPC errors in more places, some concentrated at beginning of load, some concentrated at end, some sprinkled in the middle."
git-svn-id: file:///svn/toku/tokudb@20025 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
8fe975666d loader create close hits an assert refs[t:2588] #2588
git-svn-id: file:///svn/toku/tokudb@19999 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
2146a8c2f7 add a simple test that exibits the loader->abort memory leaks refs[t:2586] #2586
git-svn-id: file:///svn/toku/tokudb@19997 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Yoni Fogel
034da281ee [t:2499]Added TOKUDB_CANCELED error code for use in canceling during poll function of loader.
git-svn-id: file:///svn/toku/tokudb@19950 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Rich Prohaska
5a3875a5ea cleanup old loader temp files when the env is opened closes[t:2550]
git-svn-id: file:///svn/toku/tokudb@19941 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Yoni Fogel
451f3e57fc [t:2499] Windows port.
ECANCELED does not exist in windows
made some functions static

git-svn-id: file:///svn/toku/tokudb@19939 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Barry Perlman
b9604f998e [t:2499] Set errno so that brtloader's bl_fwrite() function gets correct error return if the real fwrite() produces an error.
git-svn-id: file:///svn/toku/tokudb@19934 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
a9b7c490d7 [t:2499] Increase number of rows by 500X to induce creation of temp files.
git-svn-id: file:///svn/toku/tokudb@19930 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
a4eedcb492 Closes [t:2577] Fixed unitialized variable.
git-svn-id: file:///svn/toku/tokudb@19914 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
dd51442ed6 [t:2499] Verify loader-created iname files are deleted on recovery.
git-svn-id: file:///svn/toku/tokudb@19904 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
e534524e76 [t:2499] Comment changes only.
git-svn-id: file:///svn/toku/tokudb@19903 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Bradley C. Kuszmaul
28cc5d1ed9 Merge the changes from 2499d onto the main line. Fixes #2499. close[t:2499].
{{{
svn merge -r 19523:19895 https://svn.tokutek.com/tokudb/toku/tokudb.2499d
}}}
.


git-svn-id: file:///svn/toku/tokudb@19902 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Barry Perlman
82053e53d3 [t:2499] Verify no temp files before loader is run, print data_dir listing only if verbose
git-svn-id: file:///svn/toku/tokudb@19901 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Barry Perlman
84b65004df [t:2499] Add recover-loader-test
git-svn-id: file:///svn/toku/tokudb@19900 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Rich Prohaska
c37f8a21a6 fix the globals test refs[t:2449]
git-svn-id: file:///svn/toku/tokudb@19882 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Barry Perlman
67c439d87d [t:2499] Added induced failure of write() with simulated enospc to loader-cleanup-test and made corresponding change in Makefile to allow use of otherwise deprecated write() system call.
git-svn-id: file:///svn/toku/tokudb@19828 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Barry Perlman
0fc32fb911 [t:2449] Export test function (override loader's fwrite) via env. Fix bug in brtloader's handling of error from overridden fwrite().
Fix bug in ydb's handling of ephemeral loader when unable to write to disk.
Improve loader-cleanup-test.

git-svn-id: file:///svn/toku/tokudb@19825 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Rich Prohaska
640bb6fd5b c++ize the ydb and locktree include files refs[t:2511]
git-svn-id: file:///svn/toku/tokudb@19808 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00
Barry Perlman
db00df2c42 [t:2499] Added preliminary ENOSPC testing.
git-svn-id: file:///svn/toku/tokudb@19780 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:08 -04:00