Commit graph

76 commits

Author SHA1 Message Date
Yoni Fogel
6120239edf Refs Tokutek/mongo#650 Remove fullhash from rollback_log_nodes; fixes issue with large (or childful) transactions during recovery due to hash_id and filenum being decoupled 2013-11-04 22:17:01 -05:00
Rich Prohaska
2796c2d35c tokutek/ft-index#76 add US Patent 8,489,638 2013-10-04 16:49:53 -04:00
Yoni Fogel
bf4c67f20d refs #46, LOTS of refactoring done.
Isolate mempool and OMT into a new class, bndata.
Remove key from the leafentry.
2013-09-25 11:07:30 -04:00
Yoni Fogel
4ed0e8ed62 Merge branch '6712' 2013-07-17 15:09:47 -07:00
Yoni Fogel
260df68320 Refs trac [t:6712], Refs Tokutek/ft-engine#40 When possible, optimize by only checking left or right bound during a bounded query 2013-07-17 14:55:23 -07:00
Yoni Fogel
6a74f0aacd Refs Tokutek/ft-engine/#40 Add c_restrict_to_range function to force cursor to keep track of prelocked range 2013-07-17 14:55:23 -07:00
John Esmet
9858bf38c0 refs #5770 Only check one basement node on pin, remove the assumption that adjacent
available nodes are query-able.
2013-07-15 11:36:31 -04:00
Leif Walsh
77c7cdd7fe implement db->get_key_after_bytes closes #18
squashed commits:

create db->get_key_after_bytes() api down to the ft layer, and start a unit test
refs #18

setup/teardown for get_key_after_bytes test
refs #18

rename test because it really is going to be a small unit test
refs #18

implement a few initial checks
refs #18

adding actually_skipped parameter because we will want it in splitVector
refs #18

NULL -> nullptr, I'm rusty from mongo work
refs #18

fix some old apis to fit what get_key_after_bytes needs:

make keyrange_compare_s hold an FT instead of FT_HANDLE to prepare for reuse refs #18

make keyrange_compare_s hold a const dbt refs #18

fix const-correctness of fill_bfe functions refs #18

add unit test for an empty dictionary
refs #18

implement get_key_after_bytes for a single basement node, passes simple tests
refs #18

add test for multiple basement nodes
refs #18

fix usage of ft_search_t, we do need a full one after all
refs #18

check actually_skipped for correctness (even though it's an estimate)
refs #18

restructure test to be faster, and test keys before the beginning of the table
refs #18

don't try to read in the right basement node, just check the bns you have
refs #18

implement get_key_after_bytes for height > 0 trees
refs #18

return the amount skipped even if we hit the end of the table
refs #18

add inexact test for height > 0 trees, adjust constants so it's faster
refs #18

don't do I/O to bring in basement nodes just for get_key_after_bytes
refs #18

don't blindly check the first basement node, it's not guaranteed to be there anymore
refs #18

fix leak in get_key_after_bytes_unit.tdb
refs #18

add get_key_after_bytes to test_stress5
refs #18

switch to a callback API to save mallocs and memcpys
refs #18
2013-05-24 18:02:29 -04:00
Leif Walsh
82970ad7d1 #6 rename config.h to toku_config.h to prevent name clash 2013-04-27 13:52:40 -04:00
Leif Walsh
9f99932a76 refs #6566 fix typo in license text
git-svn-id: file:///svn/toku/tokudb@55577 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:36 -04:00
Leif Walsh
9fd20e61a4 refs #6566 add license text to all C/C++ files as a comment near the top
git-svn-id: file:///svn/toku/tokudb@55574 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:35 -04:00
Leif Walsh
84fef672e4 refs #6566 update all copyright strings to reflect an end year of 2013
git-svn-id: file:///svn/toku/tokudb@55567 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:35 -04:00
Zardosht Kasheff
73b86e51c4 refs #6300, fix on main
git-svn-id: file:///svn/toku/tokudb@54569 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:31 -04:00
Yoni Fogel
b77c165c15 fixes #6086 Merge 6086 to main. We now read in basement nodes if a full keyrange resides in it so that we can answer more accurately.
git-svn-id: file:///svn/toku/tokudb@54342 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:30 -04:00
Zardosht Kasheff
c43e61d621 refs #5973, implement the fractal tree pieces on main
git-svn-id: file:///svn/toku/tokudb@53585 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:30 -04:00
Zardosht Kasheff
d865cbdeb5 refs #5737, merge to main
git-svn-id: file:///svn/toku/tokudb@53577 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:30 -04:00
John Esmet
bde258f5dd refs #5918 break down serialize/compress etc times into leaf vs nonleaf
git-svn-id: file:///svn/toku/tokudb@52331 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:27 -04:00
John Esmet
49425d163b closes #5864 add compression, decompression, serialization, deserialization statistics to engine status. all interesting code paths should be covered.
git-svn-id: file:///svn/toku/tokudb@51729 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
Leif Walsh
8a80359de1 refs #5868 removing a little cruft I found
git-svn-id: file:///svn/toku/tokudb@51721 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:26 -04:00
John Esmet
b604b9d6fc fixes #5762 oldest_referenced_xid and oldest_referenced_xid_known now share a prefix, which makes searching for them together a little easier
git-svn-id: file:///svn/toku/tokudb@51667 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
John Esmet
ea0736aa1c fixes #5723 refs #5801 merge 5723 to main and merge some last-minute 5801 fixes:
- widely adopt the use of ybt abstractions instead of raw dbt management. TODO: the loader and the descriptor are still manually managed.
- remove templates from the perf framework in favor of explicit key size checks that more accurately describe "How Keys/Vals Work", and prevent type related bugs.
- also: removed some dead code, refs #5101



git-svn-id: file:///svn/toku/tokudb@51665 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:25 -04:00
Zardosht Kasheff
004cf78b56 closes #5804, merge 51139, work to read in entire internal nodes into memory, compressed, as opposed to using two I/O's, to main.
git-svn-id: file:///svn/toku/tokudb@51185 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
0062aabc73 refs #5779 merge new accounting to main
git-svn-id: file:///svn/toku/tokudb@51141 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
Zardosht Kasheff
67eb050b40 closes #5773, merge to main from tokudb.5773, Merging r51027 through r51078 into '.'
git-svn-id: file:///svn/toku/tokudb@51079 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:23 -04:00
John Esmet
4233ec1e5f refs #5559 merge 5559 to main
git-svn-id: file:///svn/toku/tokudb@50812 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:21 -04:00
Leif Walsh
c653563d3e refs #5672 use a valgrind suppression to ignore the race on the clock
git-svn-id: file:///svn/toku/tokudb@50717 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
John Esmet
1f00b8ac1d refs #5710 fix typos! oops
git-svn-id: file:///svn/toku/tokudb@50560 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:20 -04:00
Zardosht Kasheff
a0eeedd5f5 refs #5700, merge to main
git-svn-id: file:///svn/toku/tokudb@50517 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
ebdf618b76 refs #5710 add accounting for flush count, number of bytes, IO time (stored as tokutime, shown in seconds), uncompressed size, and compressed size. also add these stats for when the logger writes to disk (os_pwrite, not fsync)
git-svn-id: file:///svn/toku/tokudb@50513 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
80b8bb9716 refs #5710 add counters for the number of bytes read and time spent doing IO, for the various ways you can do a fetch. improve the name of get_tokutime() and get rid of the now defunct toku_current_time_nanoseconds()
git-svn-id: file:///svn/toku/tokudb@50481 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
John Esmet
df708c4e4f refs #5710 add FT_MSG_KEYVAL_BYTES_IN to measure the amount of key/value userdata ingested at the root. this value does not count message/xid overhead so it is a good denominator when calculating write amplification
git-svn-id: file:///svn/toku/tokudb@50404 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:19 -04:00
Leif Walsh
09ff3e9c8d refs #5671 drd annotations got weird recently
git-svn-id: file:///svn/toku/tokudb@50150 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Yoni Fogel
3af91c0f51 refs #5467 merge "kill put loader, fix hot indexer freeze issue" onto main
git-svn-id: file:///svn/toku/tokudb@50137 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:17 -04:00
Leif Walsh
704131bf4b refs #5684 fix compilation
git-svn-id: file:///svn/toku/tokudb@50064 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
17d9bd691c fixes #5684 do not memset on fake_db. only set the cmp descriptor.
git-svn-id: file:///svn/toku/tokudb@50063 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
1fbd9dd387 refs #5676 this should not be on main...
git-svn-id: file:///svn/toku/tokudb@49912 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:16 -04:00
John Esmet
6002ec10c0 refsf #5351 fix a very strange warning from the optimizer
git-svn-id: file:///svn/toku/tokudb@49861 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:15 -04:00
John Esmet
65cd284834 fixes #5351 fixes #5528 merge the new locktree to main. the locktree, locktree manager, and blocking lock request api are implemented at the toplevel under locktree/ and the ydb layer has been modified to use the new API. some kruft has been deleted from various parts of the source tree and these merge also includes a change where we kill 'includes.h'
git-svn-id: file:///svn/toku/tokudb@49851 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:15 -04:00
Leif Walsh
bf70bbb898 refs #5418 merge promotion to main
git-svn-id: file:///svn/toku/tokudb@49697 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:14 -04:00
Leif Walsh
774afe534d refs #5592 move circular_buffer, omt, threadpool, growable_array, rwlock, frwlock, kibbutz, mempool, partitioned_counter, nb_mutex, sort to util/, and some other cleanup stuff
git-svn-id: file:///svn/toku/tokudb@48763 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:13 -04:00
John Esmet
0bc5013e30 fixes #5423 fixes #5424 fixes #5503 fixes #5225 fsync always succeeds and returns void, no more ft panic or logger panic, no more impossible error paths
git-svn-id: file:///svn/toku/tokudb@48076 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:08 -04:00
Zardosht Kasheff
8f93bfd8d0 refs #5467, undo checkin
git-svn-id: file:///svn/toku/tokudb@47900 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:07 -04:00
Zardosht Kasheff
5dd0e5e897 refs #5467, merge to main
git-svn-id: file:///svn/toku/tokudb@47870 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Leif Walsh
5aa3577fed closes #5464 merge partitioned counters as ft-ops.cc status variables to main
git-svn-id: file:///svn/toku/tokudb@47804 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Leif Walsh
394662e4bc closes #5449 merge removal of __attribute__((unpacked)) from ftnode_partition to main
git-svn-id: file:///svn/toku/tokudb@47783 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Yoni Fogel
616ba2fbf5 refs #5222 Implement block cloning for rollback log nodes
git-svn-id: file:///svn/toku/tokudb@47585 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:06 -04:00
Zardosht Kasheff
06df52d6e1 refs #5442, merge to main
git-svn-id: file:///svn/toku/tokudb@47493 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:05 -04:00
Zardosht Kasheff
d59b9e772d refs #4937, merge to main
git-svn-id: file:///svn/toku/tokudb@47083 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:04 -04:00
Leif Walsh
ca7781d92d refs #5378 merge to main
git-svn-id: file:///svn/toku/tokudb@47027 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:04 -04:00
Zardosht Kasheff
3a887ea75f refs #5312, merge to main
git-svn-id: file:///svn/toku/tokudb@47022 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-17 00:01:03 -04:00