Recovery System

Recovery
03/30/2005
Yan Huang - CSCI5330 Database Implementation Recovery
Integrity or correctness of data
Would like data to be accurate or correct at all times EMP
Name
Age
White 52 Green 3421 Gray 1
13/30/2005
Integrity or consistency constraints
Predicates data must satisfy Examples:

- x is key of relation R - x y holds in R - Domain(x) = {Red, Blue, Green} - a is valid index for attribute x of R - no employee should make more than twice the average salary
13/30/2005
Definition:

Consistent state: satisfies all constraints Consistent DB: DB in consistent state
13/30/2005
Observation: DB cannot be consistent always! Example: a1 + a2 +. an = TOT (constraint) Deposit $100 in a2: a2 a2 + 100 TOT TOT + 100
13/30/2005
Example: a1 + a2 +. an = TOT (constraint) Deposit $100 in a2: a2 a2 + 100 TOT TOT + 100
a2
. . 50
. . 1000
. . 150
. . 1000
. . 150
. . 1100
TOT
13/30/2005
Transaction: collection of actions that preserve consistency
Consistent DB
Consistent DB
13/30/2005
How can we prevent/fix violations?

cc: due to data sharing only recovery: due to failures only
13/30/2005
Will not consider:

How to write correct transactions How to write correct DBMS Constraint checking & repair
That is, solutions studied here do not need to know constraints
13/30/2005
Storage hierarchy
x Memory
x Disk
13/30/2005
Operations:

Input (x): block with x memory Output (x): block with x disk
Read (x,t): do input(x) if necessary t value of x in block Write (x,t): do input(x) if necessary value of x in block t
13/30/2005
Key problem Unfinished transaction
Example
Constraint: A=B T1: A A 2 B B2
13/30/2005
T1:
Read (A,t); t t2 Write (A,t); Read (B,t); t t2 Write (B,t); Output (A); Output (B);
failure!
A: 8 16 B: 8 16
memory
13/30/2005
A: 8 16 B: 8
disk
Undo logging
T1: Read (A,t); t t2 Write (A,t); Read (B,t); t t2 Write (B,t); Output (A); Output (B); A=B
A:8 16 B:8 16
memory
13/30/2005
A:8 16 B:8 16
disk
<T1, start> <T1, A, 8> <T1, B, 8> <T1, commit> log
One complication
Log is first written in memory Not written to disk on every action

memory
A: 8 16 B: 8 16 Log: <T1,start> <T1, A, 8> <T1, B, 8>
A: 8 16 B: 8
DB Log
BAD STATE #1
13/30/2005
One complication
Log is first written in memory Not written to disk on every action

memory
A: 8 16 B: 8 16 Log: <T1,start> <T1, A, 8> <T1, B, 8> <T1, commit>

13/30/2005
A: 8 16 B: 8
DB Log
BAD STATE #2
<T1, B, 8> <T1, commit>
...
Undo logging rules
(1) For every action generate undo log record (containing old value) (2) Before x is modified on disk, log records pertaining to x must be on disk (write ahead logging: WAL) (3) Before commit is flushed to log, all writes of transaction must be reflected on disk
13/30/2005
Recovery rules:
Undo logging
(1) Let S = set of transactions with <Ti, start> in log, but no <Ti, commit> (or <Ti, abort>) record in log (2) For each <Ti, X, v> in log, in reverse order (latest earliest) do: - if Ti S then - write (X, v) - output (X)
(3) For each Ti S do - write <Ti, abort> to log
13/30/2005
What if failure during recovery? No problem! Undo idempotent
13/30/2005
To discuss:

Redo logging Undo/redo logging, why both? Checkpoints
13/30/2005
Redo logging (deferred modification)
T1:
Read(A,t); t t2; write (A,t); Read(B,t); t t2; write (B,t); Output(A); Output(B) <T1, start> <T1, A, 16> <T1, B, 16> <T1, commit> LOG
A: 8 16 B: 8 16
memory
13/30/2005
output
A: 8 16 B: 8
DB
Redo logging rules
(1) For every action, generate redo log record (containing new value) (2) Before anything is modified on disk (DB), all log records for transaction that modify things (including commit) must be on disk
13/30/2005
Recovery rules:
Redo logging
(1) Let S = set of transactions with <Ti, commit> in log (2) For each <Ti, X, v> in log, in forward order (earliest latest) do: - if Ti S then Write(X, v) Output(X)
13/30/2005
Recovery is very, very
SLOW !
Redo log:
...
...
...
First Record (1 year ago)
T1 wrote A,B Committed a year ago --> STILL, Need to redo after crash!!
Last Record
Crash
13/30/2005
Undo Recovery Better?
The first record without commit or abort should not be too far away from crash
...
...
...
first record without commit or abort
But immediate update itself is expensive

13/30/2005 Yan Huang - CSCI5330 Database Implementation Recovery
Solution: Checkpoint
(simple version)
Periodically: (1) Do not accept new transactions (2) Wait until all transactions finish (3) Flush all log records to disk (log) (4) Flush all buffers to disk (DB) (do not discard buffers) (5) Write checkpoint record on disk (log) (6) Resume transaction processing
13/30/2005
Example: what to do at recovery?
Redo log (disk):

<T1,commit>
<T2,commit> <T3,C,21> <T1,A,16> Checkpoint <T2,B,17>
...
...
...
...
...
...
Crash
13/30/2005
Undo log (disk):

<T1,commit>
<T2,commit> <T3,C,21> Checkpoint <T2,B,17> <T1,A,8>
...
...
...
...
...
...
Crash
13/30/2005
Nonquiescent Checkingpointing

Quiescent checkpointing stalls DB Nonquiescent checkpointing admits new transactions while checkpointing
13/30/2005
Nonquiescent Checkpoint Rules - undo
Write and flush a log record <START CKPT(T1, , Tk)> where T1,,Tk are active transactions When all T1, , Tk have completed, write and flush a log record <END CKPT>
13/30/2005
Undo log (disk):

<CKPT(T1,T2)>
<T2,commit>
<T1,commit>
<END CKPT>
<T2,B,12>
<T3,C,21>
<T3,B,17>
<T1,A,8>
...
...
...
...
Crash
Crash after end of checkpoint
Only need to undo all the incomplete transactions started after <START CKPT>
Undo log (disk):

<CKPT(T1,T2)>
<T2,commit>
<T1,commit>
<T2,B,12>
<T3,C,21>
<T3,B,17>
<T1,A,8>
...
...
...
...
Crash
Crash before end of checkpoint
Only need to undo all incomplete transactions started after <START CKPT> and those in <CKPT ()>
Nonquiescent Checkpoint Rules - redo
Write and flush a log record <START CKPT(T1, , Tk) where T1,,Tk are active transactions Flush all the updates of the transactions committed before <START CKPT(T1,,Tk)> Write and flush a log record <END CKPT>
13/30/2005
Redo log (disk):

<CKPT(T1,T2)>
<T2,commit>
<T1,commit>
<END CKPT>
<T2,B,12>
<T3,C,21>
<T3,B,17>
<T1,A,8>
...
...
...
...
Crash
Crash after end of checkpoint
Only need to redo all the transactions committed after the latest <START CKPT>
Redo log (disk):

<CKPT(T1,T2)>
<T2,commit>
<T1,commit>
<T2,B,12>
<T3,C,21>
<T3,B,17>
<T1,A,8>
...
...
...
...
Crash
Crash before end of checkpoint
Only need to redo all the transactions committed after the latest successful <START 13/30/2005 Yan Huang - CSCI5330 Database CKPT> Implementation Recovery
Key drawbacks:
Undo logging: need to update immediately, increasing I/O Redo logging: need to keep all modified blocks in memory until commit
13/30/2005
Update <Ti, Xid, New X val, Old X val>
Solution: undo/redo logging!
13/30/2005
Rules

Page X can be flushed before or after Ti commit Log record flushed before corresponding updated page (WAL) Flush log at commit
13/30/2005
Non-quiesce checkpoint
L O G
... for undo ...
Start-ckpt active TR: Ti,T2,...
...
Flush dirty buffers
end ckpt
...
13/30/2005
Examples what to do at recovery time?

no T1 commit
L O G
...
T1,a
...
Ckpt T1
...
Ckpt end
...
T 1b
Undo T1 (undo a,b)
13/30/2005
Example
L O G
ckpt-s T1 T1 Ckpt T1 T1 ... ... T1 ... ... ... ... ... a b end c cmt
Redo T1: (redo b,c)
13/30/2005
Examples what to do at recovery time?

no T1 commit
L O G
...
T1,a
...
Ckpt T1
...
...
T 1b
Undo T1 (all the actions)
13/30/2005
Example
L O G
ckpt-s T1 T1 ... ... T1 ... ... a b
T1 T1 ... ... ... c cmt
Redo T1: (redo b,c, a and the actions after last successful ckp-s)
13/30/2005
Recovery process:
Backwards pass (end of log -> latest checkpoint start)

construct set S of committed transactions undo actions of transactions not in S follow undo chains for transactions in (checkpoint active list) - S redo actions of S transactions
Undo pending transactions
Forward pass (latest successful checkpoint start -> end of log)
start checkpoint
13/30/2005
backward pass
forward pass
Recovery with Checkpoint Summary
Logging still obey the logging rules
undo, redo, undo-redo Undo: undo all incomplete transactions Redo: redo all completed transactions Undo-redo: combined
Recovery still obey rules

Checkpoint provides a way to limit the transactions needing undo or redo
13/30/2005
Summary
undo commit
commit means all updates on disk
WAL
redo
Commit means some updates may start to migrate to disk
Undo-redo
Commit does not signal disk update
recovery
Undo all incomplete transactions
Redo all committed transactions
Undo all incomplete transactions and redo all committed transactions
Nonquiescent ckpt
<end ckpt(T)> means every tran started before the matching <start ckpt (T)> is on disk
<end ckpt(T)> means every tran committed before the matching <start ckpt(T)> is on disk
<end ckpt(T)> means every update before the matching <start ckpt(T)> is on disk
13/30/2005
Summary

Consistency of data One source of problems: failures

- WAL
13/30/2005

Recovery System

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Recovery System

Uploaded by

Copyright:

Available Formats

Recovery

Yan Huang - CSCI5330 Database Implementation Recovery

Integrity or correctness of data

Would like data to be accurate or correct at all times EMP

White 52 Green 3421 Gray 1

Yan Huang - CSCI5330 Database Implementation Recovery

Integrity or consistency constraints

Predicates data must satisfy Examples:

Yan Huang - CSCI5330 Database Implementation Recovery

Consistent state: satisfies all constraints Consistent DB: DB in consistent state

Yan Huang - CSCI5330 Database Implementation Recovery

Yan Huang - CSCI5330 Database Implementation Recovery

Yan Huang - CSCI5330 Database Implementation Recovery

Transaction: collection of actions that preserve consistency

Yan Huang - CSCI5330 Database Implementation Recovery

How can we prevent/fix violations?

cc: due to data sharing only recovery: due to failures only

Yan Huang - CSCI5330 Database Implementation Recovery

Will not consider:

Yan Huang - CSCI5330 Database Implementation Recovery

Yan Huang - CSCI5330 Database Implementation Recovery

Yan Huang - CSCI5330 Database Implementation Recovery

Key problem Unfinished transaction

Constraint: A=B T1: A A 2 B B2

Yan Huang - CSCI5330 Database Implementation Recovery

<T1, start> <T1, A, 8> <T1, B, 8> <T1, commit> log

Log is first written in memory Not written to disk on every action

A: 8 16 B: 8 16 Log: <T1,start> <T1, A, 8> <T1, B, 8>

Yan Huang - CSCI5330 Database Implementation Recovery

Log is first written in memory Not written to disk on every action

A: 8 16 B: 8 16 Log: <T1,start> <T1, A, 8> <T1, B, 8> <T1, commit>

<T1, B, 8> <T1, commit>

Yan Huang - CSCI5330 Database Implementation Recovery

Undo logging rules

Yan Huang - CSCI5330 Database Implementation Recovery

(3) For each Ti S do - write <Ti, abort> to log

Yan Huang - CSCI5330 Database Implementation Recovery

What if failure during recovery? No problem! Undo idempotent

Yan Huang - CSCI5330 Database Implementation Recovery

Redo logging Undo/redo logging, why both? Checkpoints

Yan Huang - CSCI5330 Database Implementation Recovery

Redo logging (deferred modification)

Yan Huang - CSCI5330 Database Implementation Recovery

Redo logging rules

Yan Huang - CSCI5330 Database Implementation Recovery

Yan Huang - CSCI5330 Database Implementation Recovery

Recovery is very, very

First Record (1 year ago)

Yan Huang - CSCI5330 Database Implementation Recovery

Undo Recovery Better?

first record without commit or abort

But immediate update itself is expensive

Yan Huang - CSCI5330 Database Implementation Recovery

Example: what to do at recovery?

Redo log (disk):

Yan Huang - CSCI5330 Database Implementation Recovery

Example: what to do at recovery?

Undo log (disk):

Yan Huang - CSCI5330 Database Implementation Recovery

Yan Huang - CSCI5330 Database Implementation Recovery

Nonquiescent Checkpoint Rules - undo

Yan Huang - CSCI5330 Database Implementation Recovery

Example: what to do at recovery?

Undo log (disk):