Professional Documents
Culture Documents
Motivation
EndtoendDataIntegrity
forFileSystems
Filesystemscachea largeamountofdatainmemory
forperformance
Memorycapacityisgrowing
FromZFStoZ2FS
Filesystemsmaycachedataforalongtime
Susceptibletomemorycorruption
YupuZhang
yupu@cs.wisc.edu
Howrobustaremodernfilesystemstomemory
corruption?
AZFScasestudy
4/30/2014
Fletcher
ZFSBackground
write()
4/30/2014
ReliabilityAnalysisofZFS
read()
Questions
Whathappenswhenmemory corruptionoccurs?
Howlikelyabitflipwouldcauseproblems?
MEM
Method
DISK
Randomtest:injectbitflipsrandomly
Controlledtest:targetZFSmetadataanddata
t0
t1
Generate
t2
t3
Results
Verify
Read/writecorruptdata,systemcrash,operationfailures
Onebitfliphasnonnegligiblechance(ashighas7%)of
causingfailures
Ondiskblocksarewellprotected
Howabout blocksinmemory?
4/30/2014
4/30/2014
4/30/2014
PreviousStateoftheArt
TwoDrawbacks
Performance
EndtoendDataIntegrity
Repeatedlyaccessingdatafrominmemorycache
Strongchecksummeanshighoverhead
Checksumforeachdatablockisgeneratedandverified
byapplication
Same checksumprotectsdatathroughoutentirestack
Astrong checksumisusuallypreferred
Timeliness
Itistoolatetorecoverfromthecorruptionthatoccursbeforea
blockiswrittentodisk
Verify
Checksum
Generate
Checksum
FAIL
WritePath
ReadPath
4/30/2014
WritePath
5
4/30/2014
unbounded
time
ReadPath
6
ZettabytereliableZFS(Z2FS)
Flexible EndtoendDataIntegrity
Fletcher(slow,butstrong)
xor (fast,butweak)
write()
Cooperation fromapplication,pagecache,andFS
read()
MEM
Changechecksumacrosscomponentsorovertime
Performance
DISK
Fastbutweakerchecksumforinmemorydata
Slowbutstrongerchecksumforondiskdata
t0
Generate
Timeliness
Eachcomponentisawareofthechecksum
Verificationcancatchcorruptionintime
t1
Generate
Verify
t2
Verify
Generate
ChecksumChaining
Coverageoverlaps
Detectcorruptionintime
4/30/2014
4/30/2014
Verify
t3
Verify
vulnerablewindow!
Fletcherasdiskchecksum
xor asmemorychecksum
Betterperformance
8
4/30/2014
ZettabytereliableZFS(Z2FS)
Fletcher(slow,butstrong)
ReliabilityAnalysis
xor (fast,butweak)
write()
read()
Ananalytical framework
MEM
Modeldevices(memory,disk)andchecksums
Exploreawiderangeofreliabilityparameters
Best
Best
DISK
t0
t1
t2
t3
Consumer
Consumer
Server
Generate
Generate
Verify
Verify
ChecksumChaining
Verify
Verify
Worst
Server
Worst
Reasonaboutwhen,wheretousewhatchecksum
ShowthatZ2FSisabletoprovideZettabyte Reliability
ChecksumSwitching
Longerresidencytime=>datamorelikelybeingcorrupt
4/30/2014
4/30/2014
10
Fletcher
ReliabilityTest
PerformanceEvaluation
xor
write()
Micro&MacroBenchmark
NormalizedThroughput
MEM
DISK
t0
t1
Generate
Generate
Verify
FAIL
EndtoendZFS(Fletcher)
ZFS(static)
ZFS(dynamic)
1
0.8
0.6
0.4
0.2
0
SeqRead(cold)
SeqRead(warm)
reada1GBfile
Asktheapplicationtorewrite
webserver
WarmReadintensive
varmail
Dominately by
RandomI/Os
Betterprotectionusuallymeanshigheroverhead
Z2FShelpstoreducetheoverhead,especiallyforwarmreads
Z2FSdetectsandrecoversfromcorruptionintime
4/30/2014
ZFS
11
4/30/2014
12
4/30/2014
Summary
Endtoenddataintegrityhelpsbutisnotperfect
Slowperformance,anduntimelydetection
Onechecksumdoesnotalwaysfitall
Solution: flexibleendtoenddataintegrity
Cooperation amongcomponents
Choosethebestchecksumbasedondeviceproperties
ImplementationofZ2FS
Checksumchaining+Checksumswitching
LOC: ~6000inZFS
4/30/2014
13