John Esmet
bde258f5dd
refs #5918 break down serialize/compress etc times into leaf vs nonleaf
...
git-svn-id: file:///svn/toku/tokudb@52331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:27 -04:00
Zardosht Kasheff
f3dc3fdea2
refs #5842 , merge Direct I/O to main
...
git-svn-id: file:///svn/toku/tokudb@52238 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:27 -04:00
Zardosht Kasheff
f07a7f1e6a
closes #5554 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@51911 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
Zardosht Kasheff
ebb5315335
refs #5638 , merge to main, finally.
...
git-svn-id: file:///svn/toku/tokudb@51767 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
49425d163b
closes #5864 add compression, decompression, serialization, deserialization statistics to engine status. all interesting code paths should be covered.
...
git-svn-id: file:///svn/toku/tokudb@51729 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
0308d99b3f
fix stuff
...
git-svn-id: file:///svn/toku/tokudb@51669 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
b604b9d6fc
fixes #5762 oldest_referenced_xid and oldest_referenced_xid_known now share a prefix, which makes searching for them together a little easier
...
git-svn-id: file:///svn/toku/tokudb@51667 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
John Esmet
ea0736aa1c
fixes #5723 refs #5801 merge 5723 to main and merge some last-minute 5801 fixes:
...
- widely adopt the use of ybt abstractions instead of raw dbt management. TODO: the loader and the descriptor are still manually managed.
- remove templates from the perf framework in favor of explicit key size checks that more accurately describe "How Keys/Vals Work", and prevent type related bugs.
- also: removed some dead code, refs #5101
git-svn-id: file:///svn/toku/tokudb@51665 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
Leif Walsh
69910da437
refs #5861 fix the bug on mainline
...
git-svn-id: file:///svn/toku/tokudb@51529 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
Leif Walsh
1517b01985
refs #5672 compile drd/helgrind tests with -O1 so they complete in a reasonable amount of time
...
git-svn-id: file:///svn/toku/tokudb@51305 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:24 -04:00
John Esmet
1fd162242d
fixes #5797 cilk artifacts removed
...
git-svn-id: file:///svn/toku/tokudb@51213 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:24 -04:00
Zardosht Kasheff
004cf78b56
closes #5804 , merge 51139, work to read in entire internal nodes into memory, compressed, as opposed to using two I/O's, to main.
...
git-svn-id: file:///svn/toku/tokudb@51185 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
0062aabc73
refs #5779 merge new accounting to main
...
git-svn-id: file:///svn/toku/tokudb@51141 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
ad7254879d
refs #5773 emergency fix for transactionless environments, which do not need mvcc
...
git-svn-id: file:///svn/toku/tokudb@51115 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
b43de53fd3
closes #5783 , rename txn->recovered_from_checkpoint to txn->for_recovery
...
git-svn-id: file:///svn/toku/tokudb@51081 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
67eb050b40
closes #5773 , merge to main from tokudb.5773, Merging r51027 through r51078 into '.'
...
git-svn-id: file:///svn/toku/tokudb@51079 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
09ae64c137
refs #5768 , fix maintenence of stats numbytes during garbage collection
...
git-svn-id: file:///svn/toku/tokudb@50867 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:22 -04:00
Zardosht Kasheff
b2c3b99c81
refs #5768 , some changes
...
git-svn-id: file:///svn/toku/tokudb@50866 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:22 -04:00
Leif Walsh
a8ccc301fd
closes #5757 merge information_schema.tokudb_fractal_tree_info and information_schema.tokudb_fractal_tree_block_map to main
...
git-svn-id: file:///svn/toku/tokudb@50863 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
e04dfafca1
refs #5748 , fix small engine status bug introduced
...
git-svn-id: file:///svn/toku/tokudb@50825 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
84c2121905
refs #5748 , do gc on leaves before writing them out to disk
...
git-svn-id: file:///svn/toku/tokudb@50823 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
6d2baa7492
refs #5559 , set child->oldest_known_referenced_xid in toku_bnc_flush_to_child, remove oldest_known_referenced_xid as parameter to ft_flush_some_child
...
git-svn-id: file:///svn/toku/tokudb@50821 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
John Esmet
4233ec1e5f
refs #5559 merge 5559 to main
...
git-svn-id: file:///svn/toku/tokudb@50812 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Zardosht Kasheff
756f1ba394
refs #5744 , fix do_update
...
git-svn-id: file:///svn/toku/tokudb@50731 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Leif Walsh
c653563d3e
refs #5672 use a valgrind suppression to ignore the race on the clock
...
git-svn-id: file:///svn/toku/tokudb@50717 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
Zardosht Kasheff
58c895e17e
closes #5717 , simplify case for FT_COMMIT_ANY
...
git-svn-id: file:///svn/toku/tokudb@50673 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
John Esmet
4cc64bdbd9
refs #5710 use a sync fetch and add on the tokutime status variables, not a regular increment, since it is accessed by multiple threads
...
git-svn-id: file:///svn/toku/tokudb@50561 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
Zardosht Kasheff
efa90364ce
refs #5700 , check for existence of txn before dereferencing txn->oldest_referenced_xid
...
git-svn-id: file:///svn/toku/tokudb@50556 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
John Esmet
3c14b7e312
refs #5710 make sure this engine status variable has the closing ')'
...
git-svn-id: file:///svn/toku/tokudb@50526 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
Zardosht Kasheff
a0eeedd5f5
refs #5700 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@50517 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
ebdf618b76
refs #5710 add accounting for flush count, number of bytes, IO time (stored as tokutime, shown in seconds), uncompressed size, and compressed size. also add these stats for when the logger writes to disk (os_pwrite, not fsync)
...
git-svn-id: file:///svn/toku/tokudb@50513 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
80b8bb9716
refs #5710 add counters for the number of bytes read and time spent doing IO, for the various ways you can do a fetch. improve the name of get_tokutime() and get rid of the now defunct toku_current_time_nanoseconds()
...
git-svn-id: file:///svn/toku/tokudb@50481 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
7962622c05
refs #5710 add toku_current_time_nanosec, rename current_time_usec to use 'microsec' to be more consistent
...
git-svn-id: file:///svn/toku/tokudb@50454 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
761e2ba496
refs #5710 not the right solution. needs to ignore secondary indexes and therefore get pushed to the handlerton
...
git-svn-id: file:///svn/toku/tokudb@50450 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
df708c4e4f
refs #5710 add FT_MSG_KEYVAL_BYTES_IN to measure the amount of key/value userdata ingested at the root. this value does not count message/xid overhead so it is a good denominator when calculating write amplification
...
git-svn-id: file:///svn/toku/tokudb@50404 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
60854dacbc
refs #5710 start new instrumentation work by cleaning out some commented out instrumentation in ft-ops
...
git-svn-id: file:///svn/toku/tokudb@50403 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
Leif Walsh
0cc7d82904
refs #5418 fix typo, include config.h from toku_assert.h so that we always know whether or not to be paranoid
...
git-svn-id: file:///svn/toku/tokudb@50147 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Yoni Fogel
3af91c0f51
refs #5467 merge "kill put loader, fix hot indexer freeze issue" onto main
...
git-svn-id: file:///svn/toku/tokudb@50137 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Leif Walsh
4258288eb9
refs #5671 also don't use copyref in ft-ops.cc
...
git-svn-id: file:///svn/toku/tokudb@50010 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
44a1de0280
refs #5351 fix memory leak on main
...
git-svn-id: file:///svn/toku/tokudb@49965 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
65cd284834
fixes #5351 fixes #5528 merge the new locktree to main. the locktree, locktree manager, and blocking lock request api are implemented at the toplevel under locktree/ and the ydb layer has been modified to use the new API. some kruft has been deleted from various parts of the source tree and these merge also includes a change where we kill 'includes.h'
...
git-svn-id: file:///svn/toku/tokudb@49851 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:15 -04:00
Leif Walsh
bf70bbb898
refs #5418 merge promotion to main
...
git-svn-id: file:///svn/toku/tokudb@49697 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:14 -04:00
Zardosht Kasheff
ddfd46fb56
refs #5634 , merge bucket mutexes to main
...
git-svn-id: file:///svn/toku/tokudb@49391 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:14 -04:00
Leif Walsh
264a8e65ed
refs #5592 fix linking of util, again...
...
git-svn-id: file:///svn/toku/tokudb@48823 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:13 -04:00
Leif Walsh
774afe534d
refs #5592 move circular_buffer, omt, threadpool, growable_array, rwlock, frwlock, kibbutz, mempool, partitioned_counter, nb_mutex, sort to util/, and some other cleanup stuff
...
git-svn-id: file:///svn/toku/tokudb@48763 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:13 -04:00
Leif Walsh
5abdff33ff
refs #5464 move partitioned_counters_init/destroy to portability
...
git-svn-id: file:///svn/toku/tokudb@48690 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:12 -04:00
Leif Walsh
c3a0afeab3
refs #5368 unify race detection tool annotations in <toku_race_tools.h>
...
git-svn-id: file:///svn/toku/tokudb@48361 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:11 -04:00
John Esmet
3364bd0b0e
refs #5490 dup/nodup artifacts
...
git-svn-id: file:///svn/toku/tokudb@48249 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:10 -04:00
John Esmet
1b8c8ac4b2
refs #5500 close userdata and checkpoint userdata do not fail, so cachefile close and ft evict from memory do not fail
...
git-svn-id: file:///svn/toku/tokudb@48237 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:09 -04:00
John Esmet
c652e648fc
refs #5500 logger fsync does not fail
...
git-svn-id: file:///svn/toku/tokudb@48233 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:09 -04:00