Commit graph

360 commits

Author SHA1 Message Date
Rich Prohaska
798e923370 merge some more seq insert code to main. addresses #896
git-svn-id: file:///svn/tokudb@4627 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-18 10:22:06 +00:00
Rich Prohaska
f4ec866165 merge some seq insert code to main. addresses #896
git-svn-id: file:///svn/tokudb@4626 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-18 10:02:06 +00:00
Bradley C. Kuszmaul
6b685145d1 Merge {{{tokudb.906}}} into {{{tokudb}}}. Very few hash calls remain.
Did
{{{
svn merge -r4600:4610 https://svn.tokutek.com/tokudb/tokudb.906
}}}
and delete the 906 branch.

Fixes #906.


git-svn-id: file:///svn/tokudb@4615 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-18 00:30:36 +00:00
Bradley C. Kuszmaul
4db5b5063c Did
{{{
svn merge -r4556:4568 https://svn.tokutek.com/tokudb/tokudb.906
}}}
to incorporate the hash improvements.

Addresses #906.


git-svn-id: file:///svn/tokudb@4596 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-17 17:05:19 +00:00
Rich Prohaska
d79133ce1f bias the pivot search in favor of appends to the dictionary. addresses #896
git-svn-id: file:///svn/tokudb@4546 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-16 01:26:51 +00:00
Bradley C. Kuszmaul
9ebb4b4080 Don't always fill with zeros. Fixes #917. Gives some speedups on random.
git-svn-id: file:///svn/tokudb@4543 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-15 17:09:14 +00:00
Bradley C. Kuszmaul
a4f4d06ff9 Merge the streamlining changes from #911. Fixes #911.
git-svn-id: file:///svn/tokudb@4500 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-12 20:32:22 +00:00
Rich Prohaska
da3b69356c merge branch 838 to main. addresses #838
git-svn-id: file:///svn/tokudb@4493 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-12 13:53:39 +00:00
Rich Prohaska
68e6ca2097 need to update the cursor when skipping provisionally deleted leaf entries. addresses #889
git-svn-id: file:///svn/tokudb@4367 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-04 22:58:07 +00:00
Yoni Fogel
f9982d4a47 Addresses #855, #856.
Bugfix in integration of omt cursors.
We were considering provdel leaf entries as not deleted.

git-svn-id: file:///svn/tokudb@4343 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-03 16:45:15 +00:00
Yoni Fogel
eb41b7c0c7 Addresses #855, #856. Integrate OMT cursors with BRT for DB_PREV
git-svn-id: file:///svn/tokudb@4328 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-02 21:35:02 +00:00
Bradley C. Kuszmaul
33d04ef55f Integrate the omtcursors into the BRT. So far only DB_NEXT is implemented. Addresses #855, #856.
git-svn-id: file:///svn/tokudb@4325 c7de825b-a66e-492c-adef-691d508d4ae1
2008-06-02 20:52:12 +00:00
Vincenzo Liberatore
bf759aa8bb Closes #855 Merged back into trunk
git-svn-id: file:///svn/tokudb@4308 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-30 20:41:12 +00:00
Bradley C. Kuszmaul
5952205957 After swapping the order of the free and the alloc of the leafentry, I messed up because the alloc might move the leafentry, so the leafentry's size must be computed while the pointer is still good. Fixes #881.
git-svn-id: file:///svn/tokudb@4276 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-29 21:38:46 +00:00
Bradley C. Kuszmaul
e7150422f4 Probably repairs #880. Addresses #880.
git-svn-id: file:///svn/tokudb@4259 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-29 17:12:45 +00:00
Bradley C. Kuszmaul
0c5f094e26 Probably repairs #880. Addresses #880.
git-svn-id: file:///svn/tokudb@4258 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-29 17:05:03 +00:00
Bradley C. Kuszmaul
d4a80654d1 Simplfy the logic a little. Addresses #880.
git-svn-id: file:///svn/tokudb@4256 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-29 16:28:37 +00:00
Bradley C. Kuszmaul
8f6bbeb225 Please run the test case under the debugger (compiled with -O0), and then in gdb when it bombs do the following commands.
{{{
(gdb) print original_mp
(gdb) print last_total_size_needed
(gdb) up
(gdb) print *mp
(gdb) where
}}}


Addresses #880. 


git-svn-id: file:///svn/tokudb@4255 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-29 16:21:28 +00:00
Bradley C. Kuszmaul
e9323991d7 {{{
svn delete tokudb.872
cd tokudb
svn merge -r4187:4233 https://svn.tokutek.com/tokudb/tokudb.872
}}}

Fixes #849, #872.
Addresses #820.


git-svn-id: file:///svn/tokudb@4234 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-29 03:12:59 +00:00
Bradley C. Kuszmaul
c82a8af504 Merge the changes for #871 and #406 into the main line as
{{{
$ cd tokudb; svn merge -r 4179:4181 https://svn.tokutek.com/tokudb/tokudb.871
$ svn delete tokudb.871
}}}

Fixes #871.  Addresses #406.  (Possibly fixes both.)


git-svn-id: file:///svn/tokudb@4182 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-28 01:22:51 +00:00
Bradley C. Kuszmaul
31144bf15b Make the leaf nodes have accurate memory footprint info.
git-svn-id: file:///svn/tokudb@4178 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-27 21:08:31 +00:00
Yoni Fogel
b846ceaa9a Addresses #860
Removed extra mallocs and memcopies for toku_c_get
when the dbts use anything other than DB_DBT_USERMEM

git-svn-id: file:///svn/tokudb@4121 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-24 17:22:14 +00:00
Bradley C. Kuszmaul
fbc83b1cda Hacked fix for #832. Don't promote unless there's space for a max pivot key.
git-svn-id: file:///svn/tokudb@4105 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-23 17:04:21 +00:00
Bradley C. Kuszmaul
f60c54cad0 Fixes #22, #333 and Addresses #818:
* Create an exact bit (#818) for the sums (but doesn't calculate it.)
 * Add magic to the file header (#22)
 * Add per-subdb dupsort flags. (#333).

Did
{{{
$ cd tokudb
$ svn merge -r3990:4094 https://svn.tokutek.com/tokudb/tokudb.818/
$ cd ..
$ svn delete tokudb.818
}}}


git-svn-id: file:///svn/tokudb@4095 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-22 21:28:00 +00:00
Bradley C. Kuszmaul
0534cbcbaa Clean up the debug argument that is passed all around. Addresses #832.
git-svn-id: file:///svn/tokudb@4036 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-20 23:47:39 +00:00
Bradley C. Kuszmaul
2f47bef9e1 Add information that is printed in the #832 assertion fails. Also, don't actually assert a failure unless the node size got out of control. Thus, if the estimate is too low, print an message and proceed if possible. Addresses #832.
git-svn-id: file:///svn/tokudb@3988 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-17 00:49:13 +00:00
Bradley C. Kuszmaul
9c26f7c6f8 The proposed fix for #789. Addresses #789.
git-svn-id: file:///svn/tokudb@3983 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-16 18:48:23 +00:00
Bradley C. Kuszmaul
79df5188a4 memcmp does not accurately determine whether two keys are the same. Take out the overactive assertion. Addresses #788, #789.
git-svn-id: file:///svn/tokudb@3980 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-16 13:47:02 +00:00
Bradley C. Kuszmaul
bdc7ee0c2f Back out. Addresses #788, #789.
git-svn-id: file:///svn/tokudb@3978 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-16 13:16:29 +00:00
Bradley C. Kuszmaul
abe77e8ac2 Zardosht found what looks like the bug. This might fix the two regressions. Addresses #788, #789.
git-svn-id: file:///svn/tokudb@3973 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-15 21:27:35 +00:00
Bradley C. Kuszmaul
4aea0d2607 More tests pass for keyrange.
The same 4 cases mentioned in [3932], except now three out of four are tested:
   1. flattened nodup (tested in [3932])
   2. unflattened nodup (tested in this change set)
   3. flattened dupsort (tested in this change set
   4. unflattened dupsort (untested)

Addresses #764.


git-svn-id: file:///svn/tokudb@3944 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-13 20:14:31 +00:00
Bradley C. Kuszmaul
b07dd27c62 Start testing {{{toku_brt_keyrange}}}.
There are 4 cases
 1. flattened nodup (tested in this changeset)
 1. unflattened nodup (untested)
 1. flattened dupsort (untested)
 1. unflattened dupsort (untested)


git-svn-id: file:///svn/tokudb@3931 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-13 12:14:38 +00:00
Rich Prohaska
3d0f87f4d8 compile with gcc-4.3 on debian4. addresses #804
git-svn-id: file:///svn/tokudb@3922 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-12 18:39:21 +00:00
Yoni Fogel
467c49d564 Addresses #394
Bugfix with the way data was copied out to static buffers.
Now (again) copies to cursor's temp storage if it is not a temporary cursor
if it is a db handle call, or a temporary cursor, it will use the db's temporary storage.

git-svn-id: file:///svn/tokudb@3863 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 17:38:10 +00:00
Bradley C. Kuszmaul
eec09abdf3 Optimize the case of where we replace a leafentry. (Don't delete and insert. Instead just replace. Addresses #794.
git-svn-id: file:///svn/tokudb@3859 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 15:24:53 +00:00
Bradley C. Kuszmaul
f48447c9cf Write the brt keyrange code. Untested. Addresses #764.
git-svn-id: file:///svn/tokudb@3852 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 13:05:01 +00:00
Bradley C. Kuszmaul
ed2c880ee3 Clean up, clean up, everybody clean their share. Fixes #791.
git-svn-id: file:///svn/tokudb@3848 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 07:16:26 +00:00
Bradley C. Kuszmaul
e81be86e43 Some keyrange support. Addresess #764.
git-svn-id: file:///svn/tokudb@3846 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 07:05:26 +00:00
Yoni Fogel
9abb92dc4b Addresses #394
Merge branch tokudb.394 back into trunk

git-svn-id: file:///svn/tokudb@3841 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-07 20:03:13 +00:00
Yoni Fogel
8c0fd271d5 Addresses #783
Using DB_CURRENT no longer overwrites temporary memory owned by the db handle.
Added regression test.

git-svn-id: file:///svn/tokudb@3821 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-06 20:30:24 +00:00
Bradley C. Kuszmaul
09850cc913 Hacking makefiles. Also make the filenames used by the tests work more reliably. Addresses #782.
git-svn-id: file:///svn/tokudb@3792 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-04 16:56:15 +00:00
Bradley C. Kuszmaul
c199a47fae Dramatic speedups: For {{{db-benchmark-test -x}}} (4 interations)
||                                    ||      DB size ||  logsize ||  rate    ||
|| Before (with transactions)         ||      .33GB   ||   3.0GB  ||  17477/s ||
|| No Transactions (with this fix)    ||      .43GB   ||   0GB    || 104124/s ||
|| After removing undo info (#775)    ||      .33GB   ||   1.4GB  ||  25696/s ||
|| After optimizing commit (#725)     ||      .43GB   ||   0.6GB  ||  39909/s ||

The file size is a larger, but the logs are a lot smaller.

Addresses #725, #775.

Manipulations:
{{{
$ (cd tokudb; svn merge -r3763:3778 https://svn.tokutek.com/tokudb/tokudb.725)
$ svn delete tokudb.725
}}}


git-svn-id: file:///svn/tokudb@3779 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-03 12:43:25 +00:00
Bradley C. Kuszmaul
77fe15598c Removed the undo information from the logs (to rely on the checkpoint to save a good copy in the log).
Also, don't record the fingerprint changes, since they can be recalculated independently.  (This is a biggy.  The fingerprints updates were half the log changes.)
We'll have to do something about the fingerprints of internal nodes, so some if may come back in some form.
Speeds up {{{db-benchmark-test-tokudb -x}}} from 17477/s to 25696/s, and reduces the size of the {{{bench.tokudb}}} directory (containing logs and data) from 3.3GB to 1.7GB for a 0.33GB datafile.

Manipulations as follows:
 (cd tokudb;svn merge -r3676:3763 https://svn.tokutek.com/tokudb/tokudb.740 )
 svn delete tokudb.740

Fixes #775.


git-svn-id: file:///svn/tokudb@3764 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-02 14:38:35 +00:00
Bradley C. Kuszmaul
71b56ba1e8 Calculate an estimate of the number of {{{LEAFENTRY}}}s below each node.
Increment {{{layout_version}}} to 6.
brtdump now has a {{{--nodata}}} option (to print only the metadata, but not the data)
Fixes #766.

Merged to main using
{{{
cd tokudb/tokudb
svn merge -r3713:3725 https://svn.tokutek.com/tokudb/tokudb.766
cd ..
svn delete tokudb.766
}}}


git-svn-id: file:///svn/tokudb@3728 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-30 13:23:04 +00:00
Bradley C. Kuszmaul
7523273fb8 On commit push into tree instead of root if possible. Addresses #735.
git-svn-id: file:///svn/tokudb@3668 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-27 12:34:50 +00:00
Bradley C. Kuszmaul
d5f2eec078 Add the list of txns. Addresses #735. Fixes #749.
git-svn-id: file:///svn/tokudb@3659 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-26 03:16:30 +00:00
Bradley C. Kuszmaul
ecd0b0a971 Start adding the lists. Addresses #749.
git-svn-id: file:///svn/tokudb@3647 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 14:22:05 +00:00
Bradley C. Kuszmaul
8489ae2389 Change the {{{LEAFENTRY}}} to {{{OMTVALUE}}} when using the omt, and also change the type of {{{OMTVALUE}}} to {{{void*}}} so we can have two different OMTs coexisting. Fixes #750.
git-svn-id: file:///svn/tokudb@3645 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 13:45:55 +00:00
Bradley C. Kuszmaul
0b880e866e Merge onto the main line again.
{{{
svn merge -r3571:3582 https://svn.tokutek.com/tokudb/tokudb.724
}}}


git-svn-id: file:///svn/tokudb@3583 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 04:17:28 +00:00
Bradley C. Kuszmaul
33b252772a Merge in the OMT and its integration into the main line.
{{{
$ cd tokudb
$ svn merge -r3533:3571 https://svn.tokutek.com/tokudb/tokudb.724
}}}
 
There remain a few problems with recovery, but the main line has problems with recovery, so let's bite the bullet and move forward.

Fixes #729, #724.


git-svn-id: file:///svn/tokudb@3572 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-22 20:39:50 +00:00