Yoni Fogel
6120239edf
Refs Tokutek/mongo#650 Remove fullhash from rollback_log_nodes; fixes issue with large (or childful) transactions during recovery due to hash_id and filenum being decoupled
2013-11-04 22:17:01 -05:00
Rich Prohaska
2796c2d35c
tokutek/ft-index#76 add US Patent 8,489,638
2013-10-04 16:49:53 -04:00
Yoni Fogel
bf4c67f20d
refs #46 , LOTS of refactoring done.
...
Isolate mempool and OMT into a new class, bndata.
Remove key from the leafentry.
2013-09-25 11:07:30 -04:00
Yoni Fogel
4ed0e8ed62
Merge branch '6712'
2013-07-17 15:09:47 -07:00
Yoni Fogel
260df68320
Refs trac [t:6712], Refs Tokutek/ft-engine#40 When possible, optimize by only checking left or right bound during a bounded query
2013-07-17 14:55:23 -07:00
Yoni Fogel
6a74f0aacd
Refs Tokutek/ft-engine/#40 Add c_restrict_to_range function to force cursor to keep track of prelocked range
2013-07-17 14:55:23 -07:00
John Esmet
9858bf38c0
refs #5770 Only check one basement node on pin, remove the assumption that adjacent
...
available nodes are query-able.
2013-07-15 11:36:31 -04:00
Leif Walsh
77c7cdd7fe
implement db->get_key_after_bytes closes #18
...
squashed commits:
create db->get_key_after_bytes() api down to the ft layer, and start a unit test
refs #18
setup/teardown for get_key_after_bytes test
refs #18
rename test because it really is going to be a small unit test
refs #18
implement a few initial checks
refs #18
adding actually_skipped parameter because we will want it in splitVector
refs #18
NULL -> nullptr, I'm rusty from mongo work
refs #18
fix some old apis to fit what get_key_after_bytes needs:
make keyrange_compare_s hold an FT instead of FT_HANDLE to prepare for reuse refs #18
make keyrange_compare_s hold a const dbt refs #18
fix const-correctness of fill_bfe functions refs #18
add unit test for an empty dictionary
refs #18
implement get_key_after_bytes for a single basement node, passes simple tests
refs #18
add test for multiple basement nodes
refs #18
fix usage of ft_search_t, we do need a full one after all
refs #18
check actually_skipped for correctness (even though it's an estimate)
refs #18
restructure test to be faster, and test keys before the beginning of the table
refs #18
don't try to read in the right basement node, just check the bns you have
refs #18
implement get_key_after_bytes for height > 0 trees
refs #18
return the amount skipped even if we hit the end of the table
refs #18
add inexact test for height > 0 trees, adjust constants so it's faster
refs #18
don't do I/O to bring in basement nodes just for get_key_after_bytes
refs #18
don't blindly check the first basement node, it's not guaranteed to be there anymore
refs #18
fix leak in get_key_after_bytes_unit.tdb
refs #18
add get_key_after_bytes to test_stress5
refs #18
switch to a callback API to save mallocs and memcpys
refs #18
2013-05-24 18:02:29 -04:00
Leif Walsh
82970ad7d1
#6 rename config.h to toku_config.h to prevent name clash
2013-04-27 13:52:40 -04:00
Leif Walsh
9f99932a76
refs #6566 fix typo in license text
...
git-svn-id: file:///svn/toku/tokudb@55577 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:36 -04:00
Leif Walsh
9fd20e61a4
refs #6566 add license text to all C/C++ files as a comment near the top
...
git-svn-id: file:///svn/toku/tokudb@55574 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:35 -04:00
Leif Walsh
84fef672e4
refs #6566 update all copyright strings to reflect an end year of 2013
...
git-svn-id: file:///svn/toku/tokudb@55567 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:35 -04:00
Zardosht Kasheff
73b86e51c4
refs #6300 , fix on main
...
git-svn-id: file:///svn/toku/tokudb@54569 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:31 -04:00
Yoni Fogel
b77c165c15
fixes #6086 Merge 6086 to main. We now read in basement nodes if a full keyrange resides in it so that we can answer more accurately.
...
git-svn-id: file:///svn/toku/tokudb@54342 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:30 -04:00
Zardosht Kasheff
c43e61d621
refs #5973 , implement the fractal tree pieces on main
...
git-svn-id: file:///svn/toku/tokudb@53585 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:30 -04:00
Zardosht Kasheff
d865cbdeb5
refs #5737 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@53577 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:30 -04:00
John Esmet
bde258f5dd
refs #5918 break down serialize/compress etc times into leaf vs nonleaf
...
git-svn-id: file:///svn/toku/tokudb@52331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:27 -04:00
John Esmet
49425d163b
closes #5864 add compression, decompression, serialization, deserialization statistics to engine status. all interesting code paths should be covered.
...
git-svn-id: file:///svn/toku/tokudb@51729 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
Leif Walsh
8a80359de1
refs #5868 removing a little cruft I found
...
git-svn-id: file:///svn/toku/tokudb@51721 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
b604b9d6fc
fixes #5762 oldest_referenced_xid and oldest_referenced_xid_known now share a prefix, which makes searching for them together a little easier
...
git-svn-id: file:///svn/toku/tokudb@51667 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
John Esmet
ea0736aa1c
fixes #5723 refs #5801 merge 5723 to main and merge some last-minute 5801 fixes:
...
- widely adopt the use of ybt abstractions instead of raw dbt management. TODO: the loader and the descriptor are still manually managed.
- remove templates from the perf framework in favor of explicit key size checks that more accurately describe "How Keys/Vals Work", and prevent type related bugs.
- also: removed some dead code, refs #5101
git-svn-id: file:///svn/toku/tokudb@51665 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
Zardosht Kasheff
004cf78b56
closes #5804 , merge 51139, work to read in entire internal nodes into memory, compressed, as opposed to using two I/O's, to main.
...
git-svn-id: file:///svn/toku/tokudb@51185 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
0062aabc73
refs #5779 merge new accounting to main
...
git-svn-id: file:///svn/toku/tokudb@51141 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
67eb050b40
closes #5773 , merge to main from tokudb.5773, Merging r51027 through r51078 into '.'
...
git-svn-id: file:///svn/toku/tokudb@51079 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
4233ec1e5f
refs #5559 merge 5559 to main
...
git-svn-id: file:///svn/toku/tokudb@50812 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Leif Walsh
c653563d3e
refs #5672 use a valgrind suppression to ignore the race on the clock
...
git-svn-id: file:///svn/toku/tokudb@50717 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
John Esmet
1f00b8ac1d
refs #5710 fix typos! oops
...
git-svn-id: file:///svn/toku/tokudb@50560 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
Zardosht Kasheff
a0eeedd5f5
refs #5700 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@50517 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
ebdf618b76
refs #5710 add accounting for flush count, number of bytes, IO time (stored as tokutime, shown in seconds), uncompressed size, and compressed size. also add these stats for when the logger writes to disk (os_pwrite, not fsync)
...
git-svn-id: file:///svn/toku/tokudb@50513 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
80b8bb9716
refs #5710 add counters for the number of bytes read and time spent doing IO, for the various ways you can do a fetch. improve the name of get_tokutime() and get rid of the now defunct toku_current_time_nanoseconds()
...
git-svn-id: file:///svn/toku/tokudb@50481 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
df708c4e4f
refs #5710 add FT_MSG_KEYVAL_BYTES_IN to measure the amount of key/value userdata ingested at the root. this value does not count message/xid overhead so it is a good denominator when calculating write amplification
...
git-svn-id: file:///svn/toku/tokudb@50404 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
Leif Walsh
09ff3e9c8d
refs #5671 drd annotations got weird recently
...
git-svn-id: file:///svn/toku/tokudb@50150 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Yoni Fogel
3af91c0f51
refs #5467 merge "kill put loader, fix hot indexer freeze issue" onto main
...
git-svn-id: file:///svn/toku/tokudb@50137 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Leif Walsh
704131bf4b
refs #5684 fix compilation
...
git-svn-id: file:///svn/toku/tokudb@50064 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
17d9bd691c
fixes #5684 do not memset on fake_db. only set the cmp descriptor.
...
git-svn-id: file:///svn/toku/tokudb@50063 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
1fbd9dd387
refs #5676 this should not be on main...
...
git-svn-id: file:///svn/toku/tokudb@49912 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
6002ec10c0
refsf #5351 fix a very strange warning from the optimizer
...
git-svn-id: file:///svn/toku/tokudb@49861 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:15 -04:00
John Esmet
65cd284834
fixes #5351 fixes #5528 merge the new locktree to main. the locktree, locktree manager, and blocking lock request api are implemented at the toplevel under locktree/ and the ydb layer has been modified to use the new API. some kruft has been deleted from various parts of the source tree and these merge also includes a change where we kill 'includes.h'
...
git-svn-id: file:///svn/toku/tokudb@49851 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:15 -04:00
Leif Walsh
bf70bbb898
refs #5418 merge promotion to main
...
git-svn-id: file:///svn/toku/tokudb@49697 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:14 -04:00
Leif Walsh
774afe534d
refs #5592 move circular_buffer, omt, threadpool, growable_array, rwlock, frwlock, kibbutz, mempool, partitioned_counter, nb_mutex, sort to util/, and some other cleanup stuff
...
git-svn-id: file:///svn/toku/tokudb@48763 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:13 -04:00
John Esmet
0bc5013e30
fixes #5423 fixes #5424 fixes #5503 fixes #5225 fsync always succeeds and returns void, no more ft panic or logger panic, no more impossible error paths
...
git-svn-id: file:///svn/toku/tokudb@48076 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:08 -04:00
Zardosht Kasheff
8f93bfd8d0
refs #5467 , undo checkin
...
git-svn-id: file:///svn/toku/tokudb@47900 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:07 -04:00
Zardosht Kasheff
5dd0e5e897
refs #5467 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@47870 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Leif Walsh
5aa3577fed
closes #5464 merge partitioned counters as ft-ops.cc status variables to main
...
git-svn-id: file:///svn/toku/tokudb@47804 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Leif Walsh
394662e4bc
closes #5449 merge removal of __attribute__((unpacked)) from ftnode_partition to main
...
git-svn-id: file:///svn/toku/tokudb@47783 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Yoni Fogel
616ba2fbf5
refs #5222 Implement block cloning for rollback log nodes
...
git-svn-id: file:///svn/toku/tokudb@47585 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Zardosht Kasheff
06df52d6e1
refs #5442 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@47493 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:05 -04:00
Zardosht Kasheff
d59b9e772d
refs #4937 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@47083 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:04 -04:00
Leif Walsh
ca7781d92d
refs #5378 merge to main
...
git-svn-id: file:///svn/toku/tokudb@47027 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:04 -04:00
Zardosht Kasheff
3a887ea75f
refs #5312 , merge to main
...
git-svn-id: file:///svn/toku/tokudb@47022 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:03 -04:00