Commit graph

192 commits

Author SHA1 Message Date
Barry Perlman
593253739c [t:2781] Close 2781, add test programs to verify fix.
When processing suppress_rollback log entry, associate brt with transaction.

git-svn-id: file:///svn/toku/tokudb@21894 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:20 -04:00
Yoni Fogel
a9c554b353 [t:2641] Merge DB_READCOMMITTED bugfix (r20839) to main.
git-svn-id: file:///svn/toku/tokudb@20863 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:18 -04:00
Yoni Fogel
c9b2b066dd [t:2561] Merge tokudb.2561b to main. Disabled auto-upgrade and auto-upgrade tests
git-svn-id: file:///svn/toku/tokudb@20778 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:17 -04:00
Rich Prohaska
a4955938e5 fix the print of recovery states closes[t:2579]
git-svn-id: file:///svn/toku/tokudb@19942 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:10 -04:00
Bradley C. Kuszmaul
28cc5d1ed9 Merge the changes from 2499d onto the main line. Fixes #2499. close[t:2499].
{{{
svn merge -r 19523:19895 https://svn.tokutek.com/tokudb/toku/tokudb.2499d
}}}
.


git-svn-id: file:///svn/toku/tokudb@19902 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:09 -04:00
Yoni Fogel
8c34dda452 closes[t:2440] Merge #2440 to main.
closes[t:2512] merge #2512 to main
Delete dev branch

git-svn-id: file:///svn/toku/tokudb@19439 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:06 -04:00
Yoni Fogel
1bf7a7a403 closes[t:2449] [t:2484] Merge #2449 changes to main.
Rollback logs are now checkpointed.  There are no rolltmp files.


git-svn-id: file:///svn/toku/tokudb@19167 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:05 -04:00
Yoni Fogel
667e46d60c closes[t:2483] Fix recovery bug in loader.
checkpoint_lsn written to headers made by loader are now the fsynced 'load' log entry's lsn instead of MAX_UINT64

git-svn-id: file:///svn/toku/tokudb@19136 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:05 -04:00
Yoni Fogel
98e4b12454 [t:2476] Recovery yield calls the function it is told to.
Added asserts to rwlock locks

git-svn-id: file:///svn/toku/tokudb@18779 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:04 -04:00
Bradley C. Kuszmaul
91d93e80ca Merge r18344. No actual changes. Refs #2467. [t:2467].
git-svn-id: file:///svn/toku/tokudb@18656 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:03 -04:00
Yoni Fogel
b3f5306b41 [t:2249] Merge #2249 to main.
'load' entry and rollback/recovery,
single filename for brtloader and rest of brt
test-fsync

git-svn-id: file:///svn/toku/tokudb@18617 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:03 -04:00
Rich Prohaska
8ac1902a65 recovery checks for logcursor errors refs[t:2443]
git-svn-id: file:///svn/toku/tokudb@18343 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:02 -04:00
Bradley C. Kuszmaul
296fca6bed Contrary to r18222, the new group commit code was not on the main truck.
{{{
svn merge -r17893:18056 https://svn.tokutek.com/tokudb/toku/tokudb.2370c
}}}

Refs #2370, #2385.  [t:2370] [t:2385].


git-svn-id: file:///svn/toku/tokudb@18259 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:02 -04:00
Bradley C. Kuszmaul
178820a688 Merge changes from 2216a to main (except possibly for the windows subdirectory, which Yoni will do separately).
2216a is no longer valid.
Refs #2216. [t:2216].
{{{
svn merge -r17301:18179 https://svn.tokutek.com/tokudb/toku/tokudb.2216a/src/tests
}}}



git-svn-id: file:///svn/toku/tokudb@18181 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:01 -04:00
Yoni Fogel
14d67581f0 closes[t:2391] Merge 2391 branch to main
git-svn-id: file:///svn/toku/tokudb@18153 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:01 -04:00
Yoni Fogel
adf3e7ae95 [t:2350] Added DB_NOOVERWRITE_NO_ERROR flag for DB->put.
Will return EINVAL for ENV->put_multiple.

Has same semantics as INSERT IGNORE in mysql.
(equivalently) Has same semantics as DB_NOOVERWRITE except that it returns 0 if there is a duplicate key error
instead of DB_KEYEXISTS

git-svn-id: file:///svn/toku/tokudb@17682 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:59:00 -04:00
Barry Perlman
0fecb0fd4d Addresses #2336 refs[t:2336] Undoes unnecessary bug fix, restore assert.
git-svn-id: file:///svn/toku/tokudb@17323 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:59 -04:00
Bradley C. Kuszmaul
56d2c6fdf8 Merge 2216 onto main (from 16706-17293) as
{{{
svn merge -r16706:17293 https://svn.tokutek.com/tokudb/toku/tokudb.2216
}}}

Refs #2216. [t:2216]


git-svn-id: file:///svn/toku/tokudb@17299 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:59 -04:00
Barry Perlman
ae499b39f9 Addresses #2336 refs[t:2336] Deleted overactive assert.
git-svn-id: file:///svn/toku/tokudb@17296 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:59 -04:00
Yoni Fogel
2146c3d242 [t:2318] Replace DB_DBT_TEMPMEMORY with flags==0
git-svn-id: file:///svn/toku/tokudb@17197 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:59 -04:00
Yoni Fogel
838e0e5b1c [t:2318] Merge #2318 back to main up to r17166
git-svn-id: file:///svn/toku/tokudb@17178 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:59 -04:00
Yoni Fogel
9557f27e82 Addresses #2249 [t:2249] Merge DB_ENV->put_multiple branch back onto main.
Fixed some makefile issues, ported recovery tests to use toku_hard_crash_on_purpose(), fixed db-benchmark-test to use default name
for 0th db

git-svn-id: file:///svn/toku/tokudb@16936 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:58 -04:00
Rich Prohaska
edaa294849 print checkpoint timestamps during recovery close[t:2255]
git-svn-id: file:///svn/toku/tokudb@16485 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:56 -04:00
Rich Prohaska
26ba255118 merge the recovery changes found by barry and rich during a code review to main closes[t:2231]
git-svn-id: file:///svn/toku/tokudb@16298 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:56 -04:00
Dave Wells
dec996c12a add ability to execute 2 callbacks during recovery - for [t:2028]
git-svn-id: file:///svn/toku/tokudb@16279 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:56 -04:00
Yoni Fogel
3d64b79078 Closes #2227 closes[t:2227] Change the way backwards recovery works for
fcreate/fclose/fassociate/fopen.  Files that potentially do not have a full header written will never be opened

git-svn-id: file:///svn/toku/tokudb@16238 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:55 -04:00
Yoni Fogel
e37c1ac397 Addresses #2218 refs[t:2218] Merge r1629 to main
git-svn-id: file:///svn/toku/tokudb@16130 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:55 -04:00
Rich Prohaska
5da081a536 rearrange the recovery progress refs[t:2191]
git-svn-id: file:///svn/toku/tokudb@15981 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:54 -04:00
Rich Prohaska
ebc6a25f03 describe the units of progress time refs[t:2191]
git-svn-id: file:///svn/toku/tokudb@15980 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:54 -04:00
Rich Prohaska
e73a7f83c0 test some recovery error paths closes[t:2199]
git-svn-id: file:///svn/toku/tokudb@15979 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:54 -04:00
Rich Prohaska
c0a2cb6fd2 improve recovery progress tracking refs[t:2191]
git-svn-id: file:///svn/toku/tokudb@15959 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:53 -04:00
Rich Prohaska
4f67746f66 print the number of log entries left to process during recovery refs[t:2191]
git-svn-id: file:///svn/toku/tokudb@15947 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:53 -04:00
Rich Prohaska
508adfd5a4 replace the timestamp and shutdown log entry with a comment log entry closes[t:2192]
git-svn-id: file:///svn/toku/tokudb@15946 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:53 -04:00
Rich Prohaska
e7fd532535 print recovery progress every 15 seconds closes[t:2191]
git-svn-id: file:///svn/toku/tokudb@15938 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:53 -04:00
Rich Prohaska
4c337193b6 rename __rolltmp to __tokudb_rolltmp closes[t:2184]
git-svn-id: file:///svn/toku/tokudb@15884 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:53 -04:00
Rich Prohaska
b89eb6ef9e speed up recovery by using a large cache table closes[t:2183]
git-svn-id: file:///svn/toku/tokudb@15875 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:53 -04:00
Rich Prohaska
6196e6efca cleanup recovery fprintf strings refs[t:2177]
git-svn-id: file:///svn/toku/tokudb@15870 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:53 -04:00
Yoni Fogel
972e1f6cf2 addresses #2155 refs[t:2155] Reverse part of r15848.
We run 'backward' recovery and 'forward recovery' on the item we turn around on, so do not set 'forward saw checkpoint begin' state
till we're in forward recovery.

git-svn-id: file:///svn/toku/tokudb@15849 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:52 -04:00
Yoni Fogel
3ab29cbd36 Addresses #2155 refs[t:2155] Renamed variables,
Added missing state transaction (turn around on checkpoint begin with no transactions)
Added error checking for unlink (anything but 0/ENOENT is real error)

Changes due to code review with Barry

git-svn-id: file:///svn/toku/tokudb@15848 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:52 -04:00
Rich Prohaska
0054c5cceb add sensible recovery tracing refs[t:2177]
git-svn-id: file:///svn/toku/tokudb@15845 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:52 -04:00
Yoni Fogel
6b7c5bdfff Closes #2155 closes[t:2155] Forward recovery split
* Before 'checkpoint begin'
  * fcreate
   * Will not create (treated as fopen)
   * Will not delete file
   * Will force rollback entry of fcreate (if file exists)
 * After checkpoint begin
  * fcreate
   * Will create
   * Will first delete file
   * Will force rollback entry of fcreate (if file exists)



git-svn-id: file:///svn/toku/tokudb@15844 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:52 -04:00
Yoni Fogel
7d6f6fef69 Closes #2180 closes[t:2180] Recovery forces 'fcreate' in rollback log (when replaying fcreate log entry)
git-svn-id: file:///svn/toku/tokudb@15841 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:52 -04:00
Barry Perlman
bb8cf498bd Addresses #652, #2170, #2178 refs[t:652,2170,2178] Merge from 3.0.0-logging branch with command: svn merge -r15784:HEAD https://svn.tokutek.com/tokudb/mysql.branches/3.0.0-logging/tokudb .
git-svn-id: file:///svn/toku/tokudb@15839 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:52 -04:00
Yoni Fogel
18c2c68f7c Addresses #2164 refs[t:2164] Zombie locking contract now enforced in rename/remove
Checkpoint holds entire brt open (as zombie if necessary) instead of just holding open the cachefile.

git-svn-id: file:///svn/toku/tokudb.2037b@15740 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:06 -04:00
Yoni Fogel
a4f65b1154 Comment
git-svn-id: file:///svn/toku/tokudb.2037b@15739 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:06 -04:00
Yoni Fogel
86a917a99d Closes #2160 closes[t:2160] Implement forward and backward recovery for fdelete.
fopen forward recovery: file missing (ENOENT) is not considered an error.

git-svn-id: file:///svn/toku/tokudb.2037b@15693 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:06 -04:00
Yoni Fogel
b3df0166e4 Closes #2158 closes[t:2158] Backwards recovery of fcreate does fclose (just like backwards fopen)
git-svn-id: file:///svn/toku/tokudb.2037b@15692 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:06 -04:00
Yoni Fogel
a522ceda16 Closes #2153 refs[t:2153] fopen logged only once (unless closed and reopened). Removed logger from cachefile_close,
removed txnid from logging of fopen.

git-svn-id: file:///svn/toku/tokudb.2037b@15691 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:06 -04:00
Yoni Fogel
6feac9ff0b Closes #2064 closes[t:2064] Forward recovery of fcreate sets initial descriptor in brt.
Added test (src/tests/recover-compare-db-descriptor.abortrecover) that fails before this change, succeeds with it.

git-svn-id: file:///svn/toku/tokudb.2037b@15589 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:05 -04:00
Yoni Fogel
ae5828e7b9 Closes #2058 closes[t:2058] Recovery passes a fake DB* (zeroed out DB struct) to brt
for purposes of DB comparison function.


git-svn-id: file:///svn/toku/tokudb@15583 c7de825b-a66e-492c-adef-691d508d4ae1
2013-04-16 23:58:05 -04:00