Commit graph

74 commits

Author SHA1 Message Date
John Esmet
bde258f5dd refs #5918 break down serialize/compress etc times into leaf vs nonleaf
git-svn-id: file:///svn/toku/tokudb@52331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:27 -04:00
Zardosht Kasheff
f3dc3fdea2 refs #5842, merge Direct I/O to main
git-svn-id: file:///svn/toku/tokudb@52238 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:27 -04:00
Zardosht Kasheff
f07a7f1e6a closes #5554, merge to main
git-svn-id: file:///svn/toku/tokudb@51911 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
Zardosht Kasheff
ebb5315335 refs #5638, merge to main, finally.
git-svn-id: file:///svn/toku/tokudb@51767 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
49425d163b closes #5864 add compression, decompression, serialization, deserialization statistics to engine status. all interesting code paths should be covered.
git-svn-id: file:///svn/toku/tokudb@51729 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
0308d99b3f fix stuff
git-svn-id: file:///svn/toku/tokudb@51669 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
b604b9d6fc fixes #5762 oldest_referenced_xid and oldest_referenced_xid_known now share a prefix, which makes searching for them together a little easier
git-svn-id: file:///svn/toku/tokudb@51667 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
John Esmet
ea0736aa1c fixes #5723 refs #5801 merge 5723 to main and merge some last-minute 5801 fixes:
- widely adopt the use of ybt abstractions instead of raw dbt management. TODO: the loader and the descriptor are still manually managed.
- remove templates from the perf framework in favor of explicit key size checks that more accurately describe "How Keys/Vals Work", and prevent type related bugs.
- also: removed some dead code, refs #5101



git-svn-id: file:///svn/toku/tokudb@51665 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
Leif Walsh
69910da437 refs #5861 fix the bug on mainline
git-svn-id: file:///svn/toku/tokudb@51529 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
Leif Walsh
1517b01985 refs #5672 compile drd/helgrind tests with -O1 so they complete in a reasonable amount of time
git-svn-id: file:///svn/toku/tokudb@51305 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:24 -04:00
John Esmet
1fd162242d fixes #5797 cilk artifacts removed
git-svn-id: file:///svn/toku/tokudb@51213 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:24 -04:00
Zardosht Kasheff
004cf78b56 closes #5804, merge 51139, work to read in entire internal nodes into memory, compressed, as opposed to using two I/O's, to main.
git-svn-id: file:///svn/toku/tokudb@51185 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
0062aabc73 refs #5779 merge new accounting to main
git-svn-id: file:///svn/toku/tokudb@51141 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
ad7254879d refs #5773 emergency fix for transactionless environments, which do not need mvcc
git-svn-id: file:///svn/toku/tokudb@51115 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
b43de53fd3 closes #5783, rename txn->recovered_from_checkpoint to txn->for_recovery
git-svn-id: file:///svn/toku/tokudb@51081 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
67eb050b40 closes #5773, merge to main from tokudb.5773, Merging r51027 through r51078 into '.'
git-svn-id: file:///svn/toku/tokudb@51079 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
09ae64c137 refs #5768, fix maintenence of stats numbytes during garbage collection
git-svn-id: file:///svn/toku/tokudb@50867 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:22 -04:00
Zardosht Kasheff
b2c3b99c81 refs #5768, some changes
git-svn-id: file:///svn/toku/tokudb@50866 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:22 -04:00
Leif Walsh
a8ccc301fd closes #5757 merge information_schema.tokudb_fractal_tree_info and information_schema.tokudb_fractal_tree_block_map to main
git-svn-id: file:///svn/toku/tokudb@50863 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
e04dfafca1 refs #5748, fix small engine status bug introduced
git-svn-id: file:///svn/toku/tokudb@50825 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
84c2121905 refs #5748, do gc on leaves before writing them out to disk
git-svn-id: file:///svn/toku/tokudb@50823 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
6d2baa7492 refs #5559, set child->oldest_known_referenced_xid in toku_bnc_flush_to_child, remove oldest_known_referenced_xid as parameter to ft_flush_some_child
git-svn-id: file:///svn/toku/tokudb@50821 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
John Esmet
4233ec1e5f refs #5559 merge 5559 to main
git-svn-id: file:///svn/toku/tokudb@50812 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
756f1ba394 refs #5744, fix do_update
git-svn-id: file:///svn/toku/tokudb@50731 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Leif Walsh
c653563d3e refs #5672 use a valgrind suppression to ignore the race on the clock
git-svn-id: file:///svn/toku/tokudb@50717 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
Zardosht Kasheff
58c895e17e closes #5717, simplify case for FT_COMMIT_ANY
git-svn-id: file:///svn/toku/tokudb@50673 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
John Esmet
4cc64bdbd9 refs #5710 use a sync fetch and add on the tokutime status variables, not a regular increment, since it is accessed by multiple threads
git-svn-id: file:///svn/toku/tokudb@50561 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
Zardosht Kasheff
efa90364ce refs #5700, check for existence of txn before dereferencing txn->oldest_referenced_xid
git-svn-id: file:///svn/toku/tokudb@50556 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
John Esmet
3c14b7e312 refs #5710 make sure this engine status variable has the closing ')'
git-svn-id: file:///svn/toku/tokudb@50526 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
Zardosht Kasheff
a0eeedd5f5 refs #5700, merge to main
git-svn-id: file:///svn/toku/tokudb@50517 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
ebdf618b76 refs #5710 add accounting for flush count, number of bytes, IO time (stored as tokutime, shown in seconds), uncompressed size, and compressed size. also add these stats for when the logger writes to disk (os_pwrite, not fsync)
git-svn-id: file:///svn/toku/tokudb@50513 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
80b8bb9716 refs #5710 add counters for the number of bytes read and time spent doing IO, for the various ways you can do a fetch. improve the name of get_tokutime() and get rid of the now defunct toku_current_time_nanoseconds()
git-svn-id: file:///svn/toku/tokudb@50481 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
7962622c05 refs #5710 add toku_current_time_nanosec, rename current_time_usec to use 'microsec' to be more consistent
git-svn-id: file:///svn/toku/tokudb@50454 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
761e2ba496 refs #5710 not the right solution. needs to ignore secondary indexes and therefore get pushed to the handlerton
git-svn-id: file:///svn/toku/tokudb@50450 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
df708c4e4f refs #5710 add FT_MSG_KEYVAL_BYTES_IN to measure the amount of key/value userdata ingested at the root. this value does not count message/xid overhead so it is a good denominator when calculating write amplification
git-svn-id: file:///svn/toku/tokudb@50404 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
60854dacbc refs #5710 start new instrumentation work by cleaning out some commented out instrumentation in ft-ops
git-svn-id: file:///svn/toku/tokudb@50403 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
Leif Walsh
0cc7d82904 refs #5418 fix typo, include config.h from toku_assert.h so that we always know whether or not to be paranoid
git-svn-id: file:///svn/toku/tokudb@50147 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Yoni Fogel
3af91c0f51 refs #5467 merge "kill put loader, fix hot indexer freeze issue" onto main
git-svn-id: file:///svn/toku/tokudb@50137 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Leif Walsh
4258288eb9 refs #5671 also don't use copyref in ft-ops.cc
git-svn-id: file:///svn/toku/tokudb@50010 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
44a1de0280 refs #5351 fix memory leak on main
git-svn-id: file:///svn/toku/tokudb@49965 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
65cd284834 fixes #5351 fixes #5528 merge the new locktree to main. the locktree, locktree manager, and blocking lock request api are implemented at the toplevel under locktree/ and the ydb layer has been modified to use the new API. some kruft has been deleted from various parts of the source tree and these merge also includes a change where we kill 'includes.h'
git-svn-id: file:///svn/toku/tokudb@49851 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:15 -04:00
Leif Walsh
bf70bbb898 refs #5418 merge promotion to main
git-svn-id: file:///svn/toku/tokudb@49697 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:14 -04:00
Zardosht Kasheff
ddfd46fb56 refs #5634, merge bucket mutexes to main
git-svn-id: file:///svn/toku/tokudb@49391 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:14 -04:00
Leif Walsh
264a8e65ed refs #5592 fix linking of util, again...
git-svn-id: file:///svn/toku/tokudb@48823 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:13 -04:00
Leif Walsh
774afe534d refs #5592 move circular_buffer, omt, threadpool, growable_array, rwlock, frwlock, kibbutz, mempool, partitioned_counter, nb_mutex, sort to util/, and some other cleanup stuff
git-svn-id: file:///svn/toku/tokudb@48763 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:13 -04:00
Leif Walsh
5abdff33ff refs #5464 move partitioned_counters_init/destroy to portability
git-svn-id: file:///svn/toku/tokudb@48690 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:12 -04:00
Leif Walsh
c3a0afeab3 refs #5368 unify race detection tool annotations in <toku_race_tools.h>
git-svn-id: file:///svn/toku/tokudb@48361 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:11 -04:00
John Esmet
3364bd0b0e refs #5490 dup/nodup artifacts
git-svn-id: file:///svn/toku/tokudb@48249 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:10 -04:00
John Esmet
1b8c8ac4b2 refs #5500 close userdata and checkpoint userdata do not fail, so cachefile close and ft evict from memory do not fail
git-svn-id: file:///svn/toku/tokudb@48237 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:09 -04:00
John Esmet
c652e648fc refs #5500 logger fsync does not fail
git-svn-id: file:///svn/toku/tokudb@48233 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:09 -04:00