Commit graph

552 commits

Author SHA1 Message Date
Bradley C. Kuszmaul
7759603b6f Hacking makefiles. Addresses #782.
git-svn-id: file:///svn/tokudb@3813 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-06 10:27:46 +00:00
Bradley C. Kuszmaul
e9d3b87315 Hacking makefiles. Addresses #782.
git-svn-id: file:///svn/tokudb@3793 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-04 22:07:52 +00:00
Bradley C. Kuszmaul
09850cc913 Hacking makefiles. Also make the filenames used by the tests work more reliably. Addresses #782.
git-svn-id: file:///svn/tokudb@3792 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-04 16:56:15 +00:00
Bradley C. Kuszmaul
5f0fab0802 Fix up {{{brt->txns}}} during recovery. Fixes #781.
git-svn-id: file:///svn/tokudb@3780 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-03 17:34:14 +00:00
Bradley C. Kuszmaul
c199a47fae Dramatic speedups: For {{{db-benchmark-test -x}}} (4 interations)
||                                    ||      DB size ||  logsize ||  rate    ||
|| Before (with transactions)         ||      .33GB   ||   3.0GB  ||  17477/s ||
|| No Transactions (with this fix)    ||      .43GB   ||   0GB    || 104124/s ||
|| After removing undo info (#775)    ||      .33GB   ||   1.4GB  ||  25696/s ||
|| After optimizing commit (#725)     ||      .43GB   ||   0.6GB  ||  39909/s ||

The file size is a larger, but the logs are a lot smaller.

Addresses #725, #775.

Manipulations:
{{{
$ (cd tokudb; svn merge -r3763:3778 https://svn.tokutek.com/tokudb/tokudb.725)
$ svn delete tokudb.725
}}}


git-svn-id: file:///svn/tokudb@3779 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-03 12:43:25 +00:00
Bradley C. Kuszmaul
77fe15598c Removed the undo information from the logs (to rely on the checkpoint to save a good copy in the log).
Also, don't record the fingerprint changes, since they can be recalculated independently.  (This is a biggy.  The fingerprints updates were half the log changes.)
We'll have to do something about the fingerprints of internal nodes, so some if may come back in some form.
Speeds up {{{db-benchmark-test-tokudb -x}}} from 17477/s to 25696/s, and reduces the size of the {{{bench.tokudb}}} directory (containing logs and data) from 3.3GB to 1.7GB for a 0.33GB datafile.

Manipulations as follows:
 (cd tokudb;svn merge -r3676:3763 https://svn.tokutek.com/tokudb/tokudb.740 )
 svn delete tokudb.740

Fixes #775.


git-svn-id: file:///svn/tokudb@3764 c7de825b-a66e-492c-adef-691d508d4ae1
2008-05-02 14:38:35 +00:00
Bradley C. Kuszmaul
71b56ba1e8 Calculate an estimate of the number of {{{LEAFENTRY}}}s below each node.
Increment {{{layout_version}}} to 6.
brtdump now has a {{{--nodata}}} option (to print only the metadata, but not the data)
Fixes #766.

Merged to main using
{{{
cd tokudb/tokudb
svn merge -r3713:3725 https://svn.tokutek.com/tokudb/tokudb.766
cd ..
svn delete tokudb.766
}}}


git-svn-id: file:///svn/tokudb@3728 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-30 13:23:04 +00:00
Bradley C. Kuszmaul
efea431321 Make brtdump print the root fifo. Fixes #748.
git-svn-id: file:///svn/tokudb@3672 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-27 17:54:24 +00:00
Bradley C. Kuszmaul
461b05f4b0 All commands now push into tree instead of root if possible. Fixes #735.
git-svn-id: file:///svn/tokudb@3670 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-27 15:12:25 +00:00
Bradley C. Kuszmaul
7523273fb8 On commit push into tree instead of root if possible. Addresses #735.
git-svn-id: file:///svn/tokudb@3668 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-27 12:34:50 +00:00
Bradley C. Kuszmaul
d5f2eec078 Add the list of txns. Addresses #735. Fixes #749.
git-svn-id: file:///svn/tokudb@3659 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-26 03:16:30 +00:00
Bradley C. Kuszmaul
20bb869ac0 brt2.c undo
git-svn-id: file:///svn/tokudb@3652 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 17:46:46 +00:00
Bradley C. Kuszmaul
ecd0b0a971 Start adding the lists. Addresses #749.
git-svn-id: file:///svn/tokudb@3647 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 14:22:05 +00:00
Bradley C. Kuszmaul
9f99493d4e Change the {{{LEAFENTRY}}} to {{{OMTVALUE}}} when using the omt, and also change the type of {{{OMTVALUE}}} to {{{void*}}} so we can have two different OMTs coexisting. Fixes #750.
git-svn-id: file:///svn/tokudb@3646 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 13:46:17 +00:00
Bradley C. Kuszmaul
8489ae2389 Change the {{{LEAFENTRY}}} to {{{OMTVALUE}}} when using the omt, and also change the type of {{{OMTVALUE}}} to {{{void*}}} so we can have two different OMTs coexisting. Fixes #750.
git-svn-id: file:///svn/tokudb@3645 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-25 13:45:55 +00:00
Rich Prohaska
0c34376208 print the node layout_version. closes #743
git-svn-id: file:///svn/tokudb@3627 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-24 15:44:10 +00:00
Yoni Fogel
49737f8ee2 Addresses #724
changed magic number 100 to a constant

git-svn-id: file:///svn/tokudb@3625 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-24 15:21:39 +00:00
Yoni Fogel
22ccd07053 Addresses #724
Changes to omt-test pursuant to code review.
Now tests split at |tree|.
Deleted duplicated delete test.

git-svn-id: file:///svn/tokudb@3622 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-24 14:58:20 +00:00
Zardosht Kasheff
e17a4c1d53 addresses #724
finished testing for split and merge (this checkin has boundary cases)

git-svn-id: file:///svn/tokudb@3605 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 21:10:03 +00:00
Zardosht Kasheff
bb6fc7ebe9 addresses #724
finished most of testing for split and merge
still need to test error cases such as times when ERANGE is expected

git-svn-id: file:///svn/tokudb@3604 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 21:02:55 +00:00
Yoni Fogel
fca58d6382 Addresses #724
Put in comments for what form of tests we need for split and merge.

git-svn-id: file:///svn/tokudb@3602 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 19:54:18 +00:00
Yoni Fogel
58d3edb58b Addresses #724
Tests for insert.
BUGFIX: Insert now properly returns DB_KEYEXIST if the heaviside function
returns 0 for some value in the tree.

git-svn-id: file:///svn/tokudb@3601 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 19:51:30 +00:00
Yoni Fogel
4141778bf7 Addresses #724
Fix memory leak in test.

git-svn-id: file:///svn/tokudb@3598 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 17:57:57 +00:00
Zardosht Kasheff
a1c8113c29 Addresses #724
test delete

git-svn-id: file:///svn/tokudb@3592 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 16:46:35 +00:00
Zardosht Kasheff
437fb1bf6b addresses #724
add test for set_at

git-svn-id: file:///svn/tokudb@3591 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 15:40:57 +00:00
Zardosht Kasheff
d4dfff49aa addresses #724
remove some tabs and replace with spaces in order to make consistent
with rest of file

git-svn-id: file:///svn/tokudb@3590 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 14:16:34 +00:00
Bradley C. Kuszmaul
0b880e866e Merge onto the main line again.
{{{
svn merge -r3571:3582 https://svn.tokutek.com/tokudb/tokudb.724
}}}


git-svn-id: file:///svn/tokudb@3583 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-23 04:17:28 +00:00
Bradley C. Kuszmaul
33b252772a Merge in the OMT and its integration into the main line.
{{{
$ cd tokudb
$ svn merge -r3533:3571 https://svn.tokutek.com/tokudb/tokudb.724
}}}
 
There remain a few problems with recovery, but the main line has problems with recovery, so let's bite the bullet and move forward.

Fixes #729, #724.


git-svn-id: file:///svn/tokudb@3572 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-22 20:39:50 +00:00
Rich Prohaska
cc0fc35141 bradley fixed the child transaction commit with a rollback log in an external file. closes #730
git-svn-id: file:///svn/tokudb@3566 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-22 17:09:24 +00:00
Rich Prohaska
ddd8dff149 32 bit port
git-svn-id: file:///svn/tokudb@3532 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-21 12:09:12 +00:00
Bradley C. Kuszmaul
8813e3eb3a Store the rollbacks on disk. Fixes #711. Addresses #698 (maybe fixes it).
git-svn-id: file:///svn/tokudb@3528 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-21 04:22:45 +00:00
Bradley C. Kuszmaul
ce2f9354f3 Use the filenums slower. Fixes #722.
git-svn-id: file:///svn/tokudb@3513 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-18 21:30:39 +00:00
Bradley C. Kuszmaul
6759e38875 Fixes #721.
Improve recovery for at least one case.  Addresses #703.
The logfiles that Rich built seem corrupt, so I cannot make progress on the next case.


git-svn-id: file:///svn/tokudb@3506 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-18 14:45:04 +00:00
Rich Prohaska
b928964f4b cleanup at the end of the delete_at test. closes #720
git-svn-id: file:///svn/tokudb@3505 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-18 13:59:16 +00:00
Rich Prohaska
950b199a16 32 bit port
git-svn-id: file:///svn/tokudb@3494 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-17 12:14:13 +00:00
Bradley C. Kuszmaul
85905624d4 Create code to compute the size of a rollback entry. Addresses #711.
git-svn-id: file:///svn/tokudb@3490 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-17 10:46:19 +00:00
Bradley C. Kuszmaul
8511ea7372 Log db close so that recovery will work right if the same db is opened and closed repeatedly.
Also the file numbers can thus be reused.
Don't pass the BRT into the flush commands, since the BRT may no longer be present.
Put a counter in to see how many rollback records are present.  (Addresses #698.)
Increment the file version to 4.
Fixes #545, #703.

Note: All the tests pass except
 * Many cxx tests are getting valgrind errors.  (Addresses #716.  Possibly causes #716.)
 * {{{test_log9.recover}}} fails with "Binary files ... differ".  These will presumably be fixed by #711 or #714.  (Addresses #711, #714.)
 * {{{test_log10.recover}}} fails.   There are two failures:
  1. A valgrind problem (see #718.)  (Addresses #718.  Possibly causes #718.)
  1. The "Binary files ... differ" issue.


git-svn-id: file:///svn/tokudb@3486 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-17 03:11:55 +00:00
Rich Prohaska
d1f5bdb03f the regression no longer hit the node size assert. closes #676
git-svn-id: file:///svn/tokudb@3465 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-15 14:21:24 +00:00
Bradley C. Kuszmaul
676c4dc0ee A fix that makes the {{{test_dupsort_del}}} pass. However {{{test_dupsort_del_checksum}}} still breaks under TokuDB, but not under BDB. Addesses #690.
git-svn-id: file:///svn/tokudb@3442 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-12 14:50:50 +00:00
Bradley C. Kuszmaul
eb9b6d8558 Add support to the gpma to delete an item at a particular index. Addresses #690.
git-svn-id: file:///svn/tokudb@3441 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-12 10:24:37 +00:00
Bradley C. Kuszmaul
db89cbfd5e Comment out of the printf statement for the beta. The printf would tell us if the slow and fast get out of sync. Addresses #676.
git-svn-id: file:///svn/tokudb@3406 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-10 01:04:31 +00:00
Bradley C. Kuszmaul
1a31f3dbce The fix is to #687 is that the latest value from a le_both was providing the committed value rather than the provisional value. Fixes #687
git-svn-id: file:///svn/tokudb@3404 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 21:43:13 +00:00
Bradley C. Kuszmaul
470762fd97 A little more dead code. Addresses #684.
git-svn-id: file:///svn/tokudb@3399 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 21:08:17 +00:00
Bradley C. Kuszmaul
2d22815d0a Fix up the {{{toku_serialize_brtnode_size_slow}}} to work right on non-leaf nodes. Fixes #685.
git-svn-id: file:///svn/tokudb@3393 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 19:11:15 +00:00
Bradley C. Kuszmaul
a8a3471e43 Remove dead code for expunging fifos. Fixes #684.
git-svn-id: file:///svn/tokudb@3392 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 18:52:03 +00:00
Bradley C. Kuszmaul
2102667943 Get rid of the wrong assertion. Fixes #671.
git-svn-id: file:///svn/tokudb@3388 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 17:13:13 +00:00
Rich Prohaska
0e92670173 the node size counter gets out of sync with reality. programs assert
during serialization to the disk when the real size is smaller than the
node size counter. so we are going to recompute the node size when
the node is being serialized.


addresses #676


git-svn-id: file:///svn/tokudb@3384 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 15:46:41 +00:00
Bradley C. Kuszmaul
b58a413bce {{{
svn merge -r3352:3375 https://svn.tokutek.com/tokudb/tokudb.558a
}}}
with no conflicts.

Fixes #672, #558, #680.


git-svn-id: file:///svn/tokudb@3376 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-09 02:45:27 +00:00
Bradley C. Kuszmaul
b1ecd56ca0 Get the lsn's right on {{{brtenq}}} operations. Fixes #678.
git-svn-id: file:///svn/tokudb@3350 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-08 03:20:59 +00:00
Bradley C. Kuszmaul
ab12ba3afb Implemented tests for the confused-about-provisional-deletes bug. {{{test_log6a_abort}}} works now. Fixes #677.
git-svn-id: file:///svn/tokudb@3349 c7de825b-a66e-492c-adef-691d508d4ae1
2008-04-08 02:09:19 +00:00