Commit graph

250 commits

Author SHA1 Message Date
Rich Prohaska
c1b0e91b95 new block format with a header checksum and sub block checksums. merge -c 18916:head from 2351.ptq to main refs[t:2351]
git-svn-id: file:///svn/toku/tokudb@18961 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:05 -04:00
Rich Prohaska
864812ad48 some reorganization of brt serialize. merge -r 18898:head https://svn.tokutek.com/tokudb/toku/tokudb.2351.ptq main refs[t:2351] #2351
git-svn-id: file:///svn/toku/tokudb@18919 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:05 -04:00
Yoni Fogel
c6510b33fe closes[t:2463] Merge #2463 changes (r18552,18555) to main to fix race condition
with CACHEFILE->fd

git-svn-id: file:///svn/toku/tokudb@18612 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:02 -04:00
Bradley C. Kuszmaul
178820a688 Merge changes from 2216a to main (except possibly for the windows subdirectory, which Yoni will do separately).
2216a is no longer valid.
Refs #2216. [t:2216].
{{{
svn merge -r17301:18179 https://svn.tokutek.com/tokudb/toku/tokudb.2216a/src/tests
}}}



git-svn-id: file:///svn/toku/tokudb@18181 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:01 -04:00
Yoni Fogel
a9400fb09c closes[t:2427] Fix windows build.
git-svn-id: file:///svn/toku/tokudb@18176 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:01 -04:00
Rich Prohaska
3d46ee3fd4 merge serialize.c from 3.0.4 to main refs[t:2371]
git-svn-id: file:///svn/toku/tokudb@18055 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:00 -04:00
Rich Prohaska
1c7b2bce90 merge parallel block compressor from 3.0.4 to main closes[t:2371]
git-svn-id: file:///svn/toku/tokudb@18038 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:00 -04:00
Yoni Fogel
76340644aa [t:2350] Change BRT_LAYOUT_VERSION 11->12 to support new brt message BRT_INSERT_NO_OVERWRITE
git-svn-id: file:///svn/toku/tokudb@17701 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:00 -04:00
Rich Prohaska
2cdf2fc582 add tracing for getting a handle on node serialization costs refs[t:2261]
git-svn-id: file:///svn/toku/tokudb@17123 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:58 -04:00
Yoni Fogel
a0d3657e23 closes[t:2309] [t:2308] Deprecate dup2, _dup2 (wrap with toku_dup2), and replace /dev/null with DEV_NULL_FILE
Fixes some recovery tests in windows

git-svn-id: file:///svn/toku/tokudb@17020 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:58 -04:00
Bradley C. Kuszmaul
a3a8d8cce6 Merge from 2212 branch. Fixes #2212. Fixes[t:2212].
git-svn-id: file:///svn/toku/tokudb@16106 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:55 -04:00
Bradley C. Kuszmaul
ee8d5b670e Remove about 15 lines of dead code. Refs [t:2208]
git-svn-id: file:///svn/toku/tokudb@16068 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:55 -04:00
Yoni Fogel
88d66274a1 Closes #2128 closes[t:2128] List structure renamed to toku_list, same as header, and header moved
> to toku_include/.
> This is done to avoid poisoning namespace, and to be easily included by handlerton.

git-svn-id: file:///svn/toku/tokudb@15579 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:05 -04:00
Yoni Fogel
580f5caafb Addresses #2037 refs[t:2037] Merged #2037 back into main
git-svn-id: file:///svn/toku/tokudb@15562 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:04 -04:00
Yoni Fogel
1f72e6db67 Addresses #1936 refs[t:1936] Removed log_lsn/disk_lsn from nodes, added original version and version read from disk
Backwards compatibility for version 10 upgraded to support change (read lsns into void, set original/read_from_disk versions to 10 on read)

git-svn-id: file:///svn/toku/tokudb@14457 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:01 -04:00
Barry Perlman
ee339d03aa Addresses #1987, #1936 refs[t:1987] refs[t:1936] Insert broadcast commit on upgrade from version 10 to 11, record original version in new header field. Also fixed incorrect message type for broadcast commit.
git-svn-id: file:///svn/toku/tokudb@14430 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:01 -04:00
Bradley C. Kuszmaul
b63de170d5 Merge fix for #1957 to mainline and #1884. Fixes #1957. [t:1884] close[t:1957]
git-svn-id: file:///svn/toku/tokudb@14113 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:59 -04:00
Yoni Fogel
fee19e875a Addresses #1924 refs[t:1924] Initial version of upgrade (10->11) logic is complete
git-svn-id: file:///svn/toku/tokudb@13938 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:58 -04:00
Yoni Fogel
ffbadf36c2 Addresses #1924 refs[t:1924] Clean up deserialize brtnode in preparation for upgrade logic
git-svn-id: file:///svn/toku/tokudb@13911 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:58 -04:00
Yoni Fogel
cbee9b0e49 Addresses #1924 refs[t:1924] Added upgrade logic for brt header
git-svn-id: file:///svn/toku/tokudb@13870 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:58 -04:00
Yoni Fogel
056c85329d Addresses #1924 refs[t:1924] Deleted support for obsolete disk formats
git-svn-id: file:///svn/toku/tokudb@13852 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:58 -04:00
Yoni Fogel
110754f57c Addresses #1125 Merged nested transactions from temporary merge branch into main.
Current tests fail (not regressions, they fail as of 13461)
 * {{{x1.tdbrun}}}
 * {{{test_log(2,3,4,5,6,7,8,9,10).recover}}}
 * {{{test-recover(1,2,3).tdbrun}}}
 * {{{test1324.tdbrun}}}
ULE_DEBUG disabled (defined to 0)  Can be re-enabled for test purposes (set to 1).
refs [t:1125]
Merging into the temp branch (tokudb.main_13461+1125)
{{{svn merge --accept=postpone -r 12527:13461 ../tokudb.1125 ./}}}

Merging into main
{{{svn merge --accept=postpone -r13462:13463 ../tokudb.main_13461+1125/ ./}}}


git-svn-id: file:///svn/toku/tokudb@13464 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:56 -04:00
Rich Prohaska
a255cd1672 build with icc on linux
git-svn-id: file:///svn/toku/tokudb@12742 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:54 -04:00
Yoni Fogel
bc9fd74c80 Merge 2.0.0 back into main
svn merge --accept=postpone -r 12123:12373 ../../mysql.branches/2.0.0/tokudb/

git-svn-id: file:///svn/toku/tokudb@12375 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:53 -04:00
Yoni Fogel
049c7625a3 Addresses #1765 Merge [12079] into main (pthreads-win32 for windows)
git-svn-id: file:///svn/toku/tokudb@12081 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:53 -04:00
Yoni Fogel
09944a690e Closes #1693 Zombie brts pass off responsibilities to others.
When a new brt is open, it takes over responsibilities of all existing zombie brts, and closes them.
When a brt closes, if other brts exist, it passes off responsibilities and closes immediately.  It only becomes a zombie if no other brts exist.

(Everything above is in context of a single header).

git-svn-id: file:///svn/toku/tokudb@11533 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:51 -04:00
Barry Perlman
d560f9c9a4 Addresses #1697 Incorrect value was used to set new file size
in logic that shrinks files (to free disk space) when possible.

git-svn-id: file:///svn/toku/tokudb@11340 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:50 -04:00
Yoni Fogel
115a711093 Closes #1681, Closes #1682 two-header choosing logic is more robust
version now comes before 'size' in header
Some cleanup/defined new error codes instead of '-1'

git-svn-id: file:///svn/toku/tokudb@11304 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:49 -04:00
Bradley C. Kuszmaul
17e5214663 Refs #1529.
git-svn-id: file:///svn/toku/tokudb@11223 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:48 -04:00
Bradley C. Kuszmaul
1ed3f26d65 Add #ident Id info to all the newbrt/*.c and newbrt/*.h files. Refs #1529.
git-svn-id: file:///svn/toku/tokudb@11221 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:48 -04:00
Yoni Fogel
48245c848f Addresses #1670
Serialize descriptor with each brtnode (leaf and nonleaf) 
Deserialize descriptor with each brtnode (to be thrown away). 
    Don't malloc+memdup the descriptor for brtnodes, since that would waste time.




git-svn-id: file:///svn/toku/tokudb@11200 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:48 -04:00
Yoni Fogel
1b96373aca Addresses #1670 Standardized descriptor (de)serialization.
Added versioning to descriptor.
Changing a descriptor REQUIRES the version to increase.
Version 0 is reserved for a non-descriptor db. (cannot be set).
Not yet added to brtnodes.
Still possible to change descriptor with an open brt (which would not be upgraded).


git-svn-id: file:///svn/toku/tokudb@11198 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:48 -04:00
Yoni Fogel
5c108195c3 Addresses #1603 Root fifo is gone.
git-svn-id: file:///svn/toku/tokudb@11165 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:48 -04:00
Yoni Fogel
60812ccbfe Addresses #1603 Row descriptor changed from DB->DBT to DB->DBT*
Will aid in having multiple DB handles to a single brt, which is needed to kill root fifo

git-svn-id: file:///svn/toku/tokudb@11157 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:48 -04:00
Yoni Fogel
6561bd1af8 Addresses #1510 Final merge into main. Delete #1510 branch.
svn merge -r 11048:11110 ../tokudb.1510

git-svn-id: file:///svn/toku/tokudb@11112 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:47 -04:00
Yoni Fogel
8f2b0a7e11 Addresses #1510 Merge of 1510 (so far) back into main
Several fixes are included, particulary db remove after db truncate (with descriptor) does not crash anymore.
svn merge -r 10931:11019 ../tokudb.1510

git-svn-id: file:///svn/toku/tokudb@11020 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:47 -04:00
Yoni Fogel
0b981a3796 Addresses #1639 Can now remove (unnamed) old version dbs. Test included
git-svn-id: file:///svn/toku/tokudb@10925 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:47 -04:00
Yoni Fogel
5dc29af42c Addresses #1510 windows 32bit port of [10889]
git-svn-id: file:///svn/toku/tokudb@10914 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:47 -04:00
Yoni Fogel
dab780fdbd Addresses #1510 Merge tokud.main+1510 back into main
svn merge --accept=postpone -r 10895:10898 ../tokudb.main+1510
Previously on tokudb.main+1510:
svn merge --accept=postpone -r 10317:10892 ../tokudb.1510

git-svn-id: file:///svn/toku/tokudb@10899 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:47 -04:00
Bradley C. Kuszmaul
da0efde586 Merge in 1591 and fix conflicts. Fixes #1591.
{{{
svn merge -r10512:10803 https://svn.tokutek.com/tokudb/toku/tokudb.1591
}}}


git-svn-id: file:///svn/toku/tokudb@10804 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:47 -04:00
Yoni Fogel
cab4fe72c8 Addresses #1617 Add byte-order verification to header. version/size of header is always stored in network order
version of log is always stored in network order
cleaned up toku_htod

git-svn-id: file:///svn/toku/tokudb@10723 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:46 -04:00
Yoni Fogel
4860352fb8 Addresses #1611 toku_htod32 and toku_dtoh32 replace htonl and ntohl (for internal use).
d stands for disk-byte-order (as opposed to n for network-byte-order)
Disk-byte-order set as Intel byte order for now

git-svn-id: file:///svn/toku/tokudb@10694 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:46 -04:00
Rich Prohaska
75adfb68c7 build with icl on windows. closes #1608
git-svn-id: file:///svn/toku/tokudb@10663 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:46 -04:00
Rich Prohaska
f87a582938 fix a 32/64 bit compile issue. addresses #1489
git-svn-id: file:///svn/toku/tokudb@10644 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:46 -04:00
Rich Prohaska
9d20502817 merge 1489 to main. addresses #1489
git-svn-id: file:///svn/toku/tokudb@10639 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:46 -04:00
Yoni Fogel
784f4eb22c Closes #1575 Added toku_omt_create_steal_sorted_array
git-svn-id: file:///svn/toku/tokudb@10378 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:45 -04:00
Bradley C. Kuszmaul
9bdd6eaf62 All the old recovery tests work (#1364), and are merged onto the main line.
Refs #1510, Closes #1364.
This work was done on the 1510 branch, which we will create again.


git-svn-id: file:///svn/toku/tokudb@10313 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:44 -04:00
Yoni Fogel
d9988c26b7 Closes #1557 Blocktable now has 1 lock per brt
git-svn-id: file:///svn/toku/tokudb@10254 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:44 -04:00
Yoni Fogel
7cae47dff4 Fixes #1542 h->dirty protected by the translation table lock
git-svn-id: file:///svn/toku/tokudb@10162 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:44 -04:00
Bradley C. Kuszmaul
4257a03411 Double check that uses of snprintf inside brt code are OK. Refs #1508.
git-svn-id: file:///svn/toku/tokudb@9717 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:57:42 -04:00