Commit graph

676 commits

Author SHA1 Message Date
Bradley C. Kuszmaul
eec09abdf3 Optimize the case of where we replace a leafentry. (Don't delete and insert. Instead just replace. Addresses #794.
git-svn-id: file:///svn/tokudb@3859 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 15:24:53 +00:00
Bradley C. Kuszmaul
f48447c9cf Write the brt keyrange code. Untested. Addresses #764.
git-svn-id: file:///svn/tokudb@3852 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 13:05:01 +00:00
Bradley C. Kuszmaul
ed2c880ee3 Clean up, clean up, everybody clean their share. Fixes #791.
git-svn-id: file:///svn/tokudb@3848 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 07:16:26 +00:00
Bradley C. Kuszmaul
e81be86e43 Some keyrange support. Addresess #764.
git-svn-id: file:///svn/tokudb@3846 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-08 07:05:26 +00:00
Yoni Fogel
9abb92dc4b Addresses #394
Merge branch tokudb.394 back into trunk

git-svn-id: file:///svn/tokudb@3841 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-07 20:03:13 +00:00
Yoni Fogel
8c0fd271d5 Addresses #783
Using DB_CURRENT no longer overwrites temporary memory owned by the db handle.
Added regression test.

git-svn-id: file:///svn/tokudb@3821 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-06 20:30:24 +00:00
Bradley C. Kuszmaul
09850cc913 Hacking makefiles. Also make the filenames used by the tests work more reliably. Addresses #782.
git-svn-id: file:///svn/tokudb@3792 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-04 16:56:15 +00:00
Bradley C. Kuszmaul
c199a47fae Dramatic speedups: For {{{db-benchmark-test -x}}} (4 interations)
||                                    ||      DB size ||  logsize ||  rate    ||
|| Before (with transactions)         ||      .33GB   ||   3.0GB  ||  17477/s ||
|| No Transactions (with this fix)    ||      .43GB   ||   0GB    || 104124/s ||
|| After removing undo info (#775)    ||      .33GB   ||   1.4GB  ||  25696/s ||
|| After optimizing commit (#725)     ||      .43GB   ||   0.6GB  ||  39909/s ||

The file size is a larger, but the logs are a lot smaller.

Addresses #725, #775.

Manipulations:
{{{
$ (cd tokudb; svn merge -r3763:3778 https://svn.tokutek.com/tokudb/tokudb.725)
$ svn delete tokudb.725
}}}


git-svn-id: file:///svn/tokudb@3779 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-03 12:43:25 +00:00
Bradley C. Kuszmaul
77fe15598c Removed the undo information from the logs (to rely on the checkpoint to save a good copy in the log).
Also, don't record the fingerprint changes, since they can be recalculated independently.  (This is a biggy.  The fingerprints updates were half the log changes.)
We'll have to do something about the fingerprints of internal nodes, so some if may come back in some form.
Speeds up {{{db-benchmark-test-tokudb -x}}} from 17477/s to 25696/s, and reduces the size of the {{{bench.tokudb}}} directory (containing logs and data) from 3.3GB to 1.7GB for a 0.33GB datafile.

Manipulations as follows:
 (cd tokudb;svn merge -r3676:3763 https://svn.tokutek.com/tokudb/tokudb.740 )
 svn delete tokudb.740

Fixes #775.


git-svn-id: file:///svn/tokudb@3764 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-02 14:38:35 +00:00
Bradley C. Kuszmaul
71b56ba1e8 Calculate an estimate of the number of {{{LEAFENTRY}}}s below each node.
Increment {{{layout_version}}} to 6.
brtdump now has a {{{--nodata}}} option (to print only the metadata, but not the data)
Fixes #766.

Merged to main using
{{{
cd tokudb/tokudb
svn merge -r3713:3725 https://svn.tokutek.com/tokudb/tokudb.766
cd ..
svn delete tokudb.766
}}}


git-svn-id: file:///svn/tokudb@3728 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-30 13:23:04 +00:00
Bradley C. Kuszmaul
7523273fb8 On commit push into tree instead of root if possible. Addresses #735.
git-svn-id: file:///svn/tokudb@3668 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-27 12:34:50 +00:00
Bradley C. Kuszmaul
d5f2eec078 Add the list of txns. Addresses #735. Fixes #749.
git-svn-id: file:///svn/tokudb@3659 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-26 03:16:30 +00:00
Bradley C. Kuszmaul
ecd0b0a971 Start adding the lists. Addresses #749.
git-svn-id: file:///svn/tokudb@3647 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 14:22:05 +00:00
Bradley C. Kuszmaul
8489ae2389 Change the {{{LEAFENTRY}}} to {{{OMTVALUE}}} when using the omt, and also change the type of {{{OMTVALUE}}} to {{{void*}}} so we can have two different OMTs coexisting. Fixes #750.
git-svn-id: file:///svn/tokudb@3645 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 13:45:55 +00:00
Bradley C. Kuszmaul
0b880e866e Merge onto the main line again.
{{{
svn merge -r3571:3582 https://svn.tokutek.com/tokudb/tokudb.724
}}}


git-svn-id: file:///svn/tokudb@3583 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 04:17:28 +00:00
Bradley C. Kuszmaul
33b252772a Merge in the OMT and its integration into the main line.
{{{
$ cd tokudb
$ svn merge -r3533:3571 https://svn.tokutek.com/tokudb/tokudb.724
}}}
 
There remain a few problems with recovery, but the main line has problems with recovery, so let's bite the bullet and move forward.

Fixes #729, #724.


git-svn-id: file:///svn/tokudb@3572 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-22 20:39:50 +00:00
Bradley C. Kuszmaul
ce2f9354f3 Use the filenums slower. Fixes #722.
git-svn-id: file:///svn/tokudb@3513 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-18 21:30:39 +00:00
Bradley C. Kuszmaul
8511ea7372 Log db close so that recovery will work right if the same db is opened and closed repeatedly.
Also the file numbers can thus be reused.
Don't pass the BRT into the flush commands, since the BRT may no longer be present.
Put a counter in to see how many rollback records are present.  (Addresses #698.)
Increment the file version to 4.
Fixes #545, #703.

Note: All the tests pass except
 * Many cxx tests are getting valgrind errors.  (Addresses #716.  Possibly causes #716.)
 * {{{test_log9.recover}}} fails with "Binary files ... differ".  These will presumably be fixed by #711 or #714.  (Addresses #711, #714.)
 * {{{test_log10.recover}}} fails.   There are two failures:
  1. A valgrind problem (see #718.)  (Addresses #718.  Possibly causes #718.)
  1. The "Binary files ... differ" issue.


git-svn-id: file:///svn/tokudb@3486 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-17 03:11:55 +00:00
Bradley C. Kuszmaul
676c4dc0ee A fix that makes the {{{test_dupsort_del}}} pass. However {{{test_dupsort_del_checksum}}} still breaks under TokuDB, but not under BDB. Addesses #690.
git-svn-id: file:///svn/tokudb@3442 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-12 14:50:50 +00:00
Bradley C. Kuszmaul
1a31f3dbce The fix is to #687 is that the latest value from a le_both was providing the committed value rather than the provisional value. Fixes #687
git-svn-id: file:///svn/tokudb@3404 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 21:43:13 +00:00
Bradley C. Kuszmaul
a8a3471e43 Remove dead code for expunging fifos. Fixes #684.
git-svn-id: file:///svn/tokudb@3392 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 18:52:03 +00:00
Bradley C. Kuszmaul
2102667943 Get rid of the wrong assertion. Fixes #671.
git-svn-id: file:///svn/tokudb@3388 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 17:13:13 +00:00
Bradley C. Kuszmaul
b58a413bce {{{
svn merge -r3352:3375 https://svn.tokutek.com/tokudb/tokudb.558a
}}}
with no conflicts.

Fixes #672, #558, #680.


git-svn-id: file:///svn/tokudb@3376 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 02:45:27 +00:00
Bradley C. Kuszmaul
b1ecd56ca0 Get the lsn's right on {{{brtenq}}} operations. Fixes #678.
git-svn-id: file:///svn/tokudb@3350 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-08 03:20:59 +00:00
Bradley C. Kuszmaul
ab12ba3afb Implemented tests for the confused-about-provisional-deletes bug. {{{test_log6a_abort}}} works now. Fixes #677.
git-svn-id: file:///svn/tokudb@3349 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-08 02:09:19 +00:00
Bradley C. Kuszmaul
f618aba2b8 Changing the size calculation broke brt-test. Changed it so if cachetable overflows its budget, then the cachetable continues to work. Addresses #669.
git-svn-id: file:///svn/tokudb@3338 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-07 19:52:49 +00:00
Rich Prohaska
8e0a9cc188 delete an assert that causes db-build to crash. addresses #671
git-svn-id: file:///svn/tokudb@3336 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-07 18:11:13 +00:00
Bradley C. Kuszmaul
77b41b1938 Make {{{brtnode_size()}}} more accurate. Fixes #669.
git-svn-id: file:///svn/tokudb@3334 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-07 14:53:10 +00:00
Bradley C. Kuszmaul
6298988642 Merge the tokudb.558 branch back to to the main branch with:
{{{
svn merge -r3272:3320 https://svn.tokutek.com/tokudb/tokudb.558
}}}
No conflicts.


git-svn-id: file:///svn/tokudb@3322 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-07 01:30:25 +00:00
Bradley C. Kuszmaul
7e5dfd3e5d Write leafentry code, and get things to compile. Addresses #558.
git-svn-id: file:///svn/tokudb@3269 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-04 18:03:03 +00:00
Rich Prohaska
5b9d12914e toku_verify_gpma removed as it is part of code verification. closes #633
git-svn-id: file:///svn/tokudb@3240 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-03 17:11:43 +00:00
Bradley C. Kuszmaul
f293366ddc Fix the undersized buffer pool problem. Fixes #632. Addresses #580.
git-svn-id: file:///svn/tokudb@3237 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-03 13:13:21 +00:00
Bradley C. Kuszmaul
02e0667890 Create a test for #632. Addresses #632.
git-svn-id: file:///svn/tokudb@3234 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-03 13:02:15 +00:00
Bradley C. Kuszmaul
83d416fa09 Merge the 558 branch back to the main with
{{{
svn merge -r3061:3225 https://svn.tokutek.com/tokudb/tokudb.558
}}}

Fixes #630.
Addresses #558.


git-svn-id: file:///svn/tokudb@3226 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-02 23:40:36 +00:00
Bradley C. Kuszmaul
0c1af47865 Add the first two tests for {{{DB_ENV->log_archive}}}. Fix up some memory leaks. Add {{{LSN*}}} to all the {{{toku_log_*}}} functions so that {{{toku_logger_txn_begin}}} can use the LSN as a txnid. Addresses #75, #83, #392.
git-svn-id: file:///svn/tokudb@3004 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-21 21:02:30 +00:00
Bradley C. Kuszmaul
70bb010995 make {{{DB_ENV->log_archive}}} return something (but it's not quite right). Clean up the serialization of the brt header. Fix up {{{DB_ENV->txn_checkpoint}}} (but it's not quite right). Addresses #75, #83, #392.
git-svn-id: file:///svn/tokudb@3000 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-21 19:40:32 +00:00
Bradley C. Kuszmaul
2d7c5ba201 Fix MAX_PATHLEN_TO_ROOT bug. Fixes #570.
git-svn-id: file:///svn/tokudb@2991 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-21 12:47:55 +00:00
Bradley C. Kuszmaul
401eb1d0bc test_abort2 works up until a lock tree problem. I think the test and abort mechanisms are right now. I'll close #556 but file a new bug (#569). Fixes #556. Addresses #569.
git-svn-id: file:///svn/tokudb@2982 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-20 22:01:57 +00:00
Bradley C. Kuszmaul
95398c5397 Rollback is better, but still broken. Addresses #556.
git-svn-id: file:///svn/tokudb@2960 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-19 22:42:46 +00:00
Bradley C. Kuszmaul
538a507a30 Make rollback do the right thing in some cases fo internal nodes. Addresses #556.
git-svn-id: file:///svn/tokudb@2955 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-19 19:23:45 +00:00
Bradley C. Kuszmaul
6d19bad142 Fixes many problems in {{{test_log10}}}. There's a memory leak and the recovered files still differ, however. Addresses #548.
git-svn-id: file:///svn/tokudb@2913 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-18 10:19:41 +00:00
Bradley C. Kuszmaul
522627d9ba Log deq, but still not quite passing the test. Addresses #548.
git-svn-id: file:///svn/tokudb@2898 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-17 18:56:12 +00:00
Bradley C. Kuszmaul
60b6abe1c7 {{{test_log8}}} recovers now. Fixes #544.
git-svn-id: file:///svn/tokudb@2880 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-17 02:40:59 +00:00
Bradley C. Kuszmaul
771125a0cd Finish making the valgrind error go away in test_logmax. Fixes #542.
git-svn-id: file:///svn/tokudb@2865 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-16 14:01:20 +00:00
Bradley C. Kuszmaul
99b75cabcd Deal with the multithreaded cursor DBT memory allocaiton problem. Addresses #538. (I'm going to write a few more tests before calling it fixed.)
git-svn-id: file:///svn/tokudb@2854 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-15 19:06:39 +00:00
Bradley C. Kuszmaul
0f29f9abe7 Group commit working in tokulogger. Need to release some locks in ydb. Addresses #484.
git-svn-id: file:///svn/tokudb@2763 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-12 17:55:11 +00:00
Bradley C. Kuszmaul
e037aa5cae Get DUPSORT dbs with logging to work without assertion failures. Recovery stlll is broken, however. Addresses #482. Fixes #442.
git-svn-id: file:///svn/tokudb@2602 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-07 02:06:15 +00:00
Bradley C. Kuszmaul
241c5392df Break up brt-test some more. Addresses #475. Also make the fanout flexible. Fixes #126.
git-svn-id: file:///svn/tokudb@2593 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-06 21:46:57 +00:00
Bradley C. Kuszmaul
5c52e7c64a Separate some of the test helpers from brt.c to brt-test-helpers.c. Addresses #472.
git-svn-id: file:///svn/tokudb@2556 c7de825b-a66e-492c-adef-691d508d4ae1
2008-03-05 18:34:32 +00:00
Bradley C. Kuszmaul
dd4a6feabe Improvements to rollback. Addresses #27.
git-svn-id: file:///svn/tokudb@2435 c7de825b-a66e-492c-adef-691d508d4ae1
2008-02-27 11:05:19 +00:00