1/6/2015

Bilgin'sBlog|KalmanFilterforDummies
Login|

Home

Dreamland

Retrospective

Now&Future

Bits&Bytes

About|Affiliate
Program|Sitemap

About

thisarticleisdedicatedtoone
specialfriendwithahigh
entropylife,whosekindly

AMathematicallyChallengedMan'sSearchforScientificWisdom

Supposeyouhaveasignal,anytype.

BilginEsmeMarch2009

Forexampleasound,someradar
readingsorevenadigitizedimage.

inspirationprovidedmethe

couragetopersevereto

understandoverwhelmingly
difficultsubjects,andeventually
mademepossibletocreatethis
material

Definition of the Problem

KalmanFilterForDummies

WhenIstarteddoingmyhomework

Andofcourseyouhavenoiseinthe

forOptimalFilteringforSignal

environment.Whatcanyoudoto

Processingclass,Isaidtomyself

discardthisnoise?Thesimplestthing

:"Howhardcanitbe?".SoonI

thatcomestomindis,"takingthe

realizedthatitwasafatalmistake.

averageofsomeconsequent
samples".Thissimpleapproachof

Thewholethingwaslikeanightmare.

coursedoesn'tworkformostreallife

Nothingmadesense.Theequations

problems.Weneedamore

werecomposedofsomeridiculously

sophisticatedapproach.

complexsuperscriptedand

Some Highlights
Firstofall,it'snotafilteratall,it's
anestimator.
It'savery,veryimportantthing,it's
notanoveremphasizebelieveme
Beingregardedasoneofthe

subscriptedvariablescombinedwithtransposedmatricesanduntransposedsome

DigitalSignalProcessingscholarsdeal

otherstuff,whicharetotallyunknowabletomostofus.

withthissameproblemfordecades,
andtherearelotsoftechniques

Andthen,insteadofaimingforthehomework,Idecidedfirstfullyconcentrating

developedforthisproblem.Kalman

onKalmanFilteritself.Thisarticleistheresultofmycoupleofday'sworkand

Filterisoneofthesetechniques.And

reflectstheslowlearningcurvesofa"mathematicallychallenged"person.If

averypowerfulone.

you'rehumbleenoughtoadmitthatyoudon'tunderstandthisstuffcompletely,
you'llfindthismaterialveryenlightening.
So,enjoyit!

greatestdiscoveriesin20thcentury.
Hardtomasteritcompletely,but
it'spossibletoplaywithit,withlittle
mathematicalbackground
Veryconvenienttoimplementasa
computeralgorithm

AQuickInsight
AsImentionedearlier,it'snearlyimpossibletograspthefullmeaningofKalmanFilterbystartingfromdefinitionsand
complicatedequations(atleastforusmeremortals).Formostcases,thestatematricesdropoutandweobtainthebelow
equation,whichismucheasiertostartwith.

It'sarecursivemethod,which
means,foreachinstance,youuse
thepreviousoutputasaninput.
Yes,theequationsarevery
complicated,andincludessome
mysteriousmatrices.Butmostofthe
time,youomitorignorethem
unlessyoucarrythroughreally
complicatedscience.

Remember,thek'sonthesubscriptarestates.Herewecantreatitasdiscretetimeintervals,suchask=1means1ms,k=2
means2ms.
Ourpurposeistofind
Alsohere,

,theestimateofthesignalx.Andwewishtofinditforeachconsequentk's.

isthemeasurementvalue.Keepinmindthat,wearenotperfectlysureofthesevalues.Otherwise,wewon't

beneedingtodoallthese.And

iscalled"KalmanGain"(whichisthekeypointofallthese),and

istheestimate

ofthesignalonthepreviousstate.
TheonlyunknowncomponentinthisequationistheKalmanGain

.Because,wehavethemeasurementvalues,andwe

alreadyhavethepreviousestimatedsignal.YoushouldcalculatethisKalmanGainforeachconsequentstate.Thisisnoteasy
ofcourse,butwehaveallthetoolstodoit.
Ontheotherhand,let'sassume
smarter

tobe0.5,whatdoweget?It'sasimpleaveraging!Inotherwords,weshouldfind

coefficientsateachstate.Thebottomlineis:

Kalmanfilterfindsthemostoptimumaveragingfactorforeachconsequentstate.Alsosomehowremembersa
littlebitaboutthepaststates.

http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

1/5

1/6/2015

Bilgin'sBlog|KalmanFilterforDummies
Isn'tthisamazing?

StepbyStepGuide
Here'sasimplestepbystepguideforaquickstarttoKalmanfiltering.

Rubber Boom
March2014

STEP1BuildaModel

AnswerstoSillyQuestions

It'sthemostimportantstep.Firstofall,youmustbesurethat,Kalmanfiltering

CanIdeployKalmanFilterto

conditionsfittoyourproblem.
MynewgameRubberBoomis
finallyonAppStore!

allDigitalSignalProcessing
AswerememberthetwoequationsofKalmanFilterisasfollows:

problems?
I'veseenlotsofpapersthat

Surviveandthrive
intheAmazon!

useKalmanFilterforavariety
ofproblems,suchasnoise
filtering,subspacesignalanalysis,
featureextractionandsoon.The

RubberBoomisarealtimestrategy

Itmeansthateachxk (oursignalvalues)maybeevaluatedbyusingalinear

bottomlineis,youcanuseKalman

gamethattakesplaceintheearly

stochasticequation(thefirstone).Anyxk isalinearcombinationofitsprevious

Filterwithaquiteapproximationand

20thCenturyAmazonjungle.

valueplusacontrolsignaluk andaprocessnoise(whichmaybehardto

clevermodeling.

Establishathrivingrubbercollecting

conceptualize).Rememberthat,mostofthetime,there'snocontrolsignaluk .

businessinthisfierceenvironment.

Thesecondequationtellsthatanymeasurementvalue(whichwearenotsureits

Rulesareverysimple:"Collect

accuracy)isalinearcombinationofthesignalvalueandthemeasurementnoise.

CanIuseitforImage

rubberandsellittothepassing

TheyarebothconsideredtobeGaussian.

Processing?

boats."Inthemeantime,becareful
withthewildbirds,Indians,snakes

Theprocessnoiseandmeasurementnoisearestatisticallyindependent.

andmanyotherchallenges.Planyour
routeaccordingtoyourfoodreserve

TheentitiesA,BandHareingeneralformmatrices.Butinmostofoursignal

andlookforwaystogetadditional

processingproblems,weusemodelssuchthattheseentitiesarejustnumeric

foodontheway.

values.Alsoasanadditionalease,whilethesevaluesmaychangebetweenstates,

Ofcourse.

mostofthetime,wecanassumethatthey'reconstant.
WheredowefindtheseTime
Ifweareprettysurethatoursystemfitsintothismodel(mostofthesystemsdo

UpdateandMeasurement

bytheway),theonlythingleftistoestimatethemeanandstandarddeviationof

Updateequations?Itseems

thenoisefunctionsWk1andvk .Weknowthat,inreallife,nosignalispure

thattheysuddenlyappearedfrom

Gaussian,butwemayassumeitwithsomeapproximation.Thisisnotabig

nowhere.

problem,becausewe'llseethattheKalmanFilteringAlgorithmtriestoconverge
intocorrectestimations,eveniftheGaussiannoiseparametersarepoorly

Youcanderiveitfromthe

estimated.

linearstochasticdifference
equation(theequationsin

Theonlythingtokeepinmindis:"Thebetteryouestimatethenoiseparameters,

STEP1),bytakingthepartial

thebetterestimatesyouget."

derivativeandsettingthemtozero
(forminimizingtheestimationerror).

MoreaboutRubberBoom

Ofcoursethey'rehardandtime

STEP2StarttheProcess

consuming.

IfyousucceededtofityourmodelintoKalmanFilter,thanthenextstepisto
determinethenecessaryparametersandyourinitialvalues.

Also Check

Wehavetwodistinctsetofequations:TimeUpdate(prediction)and

VectorRotationfor
Dummies!

state.

MeasurementUpdate(correction).Bothequationsetsareappliedateachkth

Maybethetitleisnotvery
appropriate,becausethevector
rotationoperationsarenotsosimple

TimeUpdate

MeasurementUpdate

(prediction)

(correction)

Rudolf Emil Kalman

innature.Youshouldtakeacoupleof
PhDlevelcoursestocompletely
understanditifyou'resimplyan
engineer,notamathematicianor
physicist.
RudolfKalmanwasbornin

Thebadnewsisasagame

Budapest,Hungary,andobtainedhis

developer,youshouldknowyour
mathquitewell.Now,relax!You'rein
betterhandsyouhopedfor.

bachelor'sdegreein1953and
WemadethemodelinginSTEP1,soweknowthematricesA,BandH.Most

master'sdegreein1954fromMITin

probably,theywillbenumericalconstants.Andevenmostprobably,they'llbe

electricalengineering.Hisdoctorate

equalto1.Isuggestyoutorewritetheseequationsandseehowsimplifiedwill

in1957wasfromColumbia

theseequationsbecome.(ifyou'relazyenoughnottodoit,I'lldoitforyouinthe

University.

http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

2/5

1/6/2015

Bilgin'sBlog|KalmanFilterforDummies
Examplebelow).
Kalmanisanelectricalengineerby
ThemostremainingpainfulthingistodetermineRandQ.Risrathersimpleto

training,andisfamousforhisco

findout,because,ingeneral,we'requitesureaboutthenoiseintheenvironment.

inventionofthe

ButfindingoutQisnotsoobvious.Andatthisstage,Ican'tgiveyouaspecific

Kalmanfilter,amathematical

method.

techniquewidelyusedincontrol
systemsandavionicstoextracta

Tostarttheprocess,weneedtoknowtheestimateofx0,andP0.

signalfromaseriesofincomplete
andnoisymeasurements.

ReadtheArticle

STEP3Iterate

Kalman'sideasonfilteringwere

Afterwegatheredalltheinformationweneedandstartedtheprocess,nowwe
caniteratethroughtheestimates.Keepinmindthatthepreviousestimateswillbe
theinputforthecurrentstate.

initiallymetwithskepticism,somuch
sothathewas
forcedtofirstpublishhisresultsina
mechanical(ratherthanelectrical)
engineeringjournal.
Hehadmoresuccessinpresenting
hisideas,however,whilevisiting
StanleyF.Schmidtatthe
NASAAmesResearchCenterin1960.
ThisledtotheuseofKalmanfilters
duringtheApolloprogram.

Here,

isthe"priorestimate"whichinaway,meanstheroughestimate

beforethemeasurementupdatecorrection.Andalso

isthe"priorerror

covariance".Weusethese"prior"valuesinourMeasurementUpdateequations.
InMeasurementUpdateequations,wereallyfind
timek(theverythingwewishtofind).Also,wefind
thek1(future)estimate,togetherwith

whichistheestimateofxat
whichisnecessaryfor

.TheKalmanGain(

)weevaluate

isnotneededforthenextiterationstep,it'sahidden,mysteriousandthemost
importantpartofthissetofequations.
ThevaluesweevaluateatMeasurementUpdatestagearealsocalled
"posterior"values.Whichalsomakessense.

ASimpleExample
Nowlet'strytoestimateascalarrandomconstant,suchasa"voltagereading"fromasource.Solet'sassumethatithasaconstantvalueofaV(volts),butofof
coursewesomenoisyreadingsaboveandbelowavolts.Andweassumethatthestandarddeviationofthemeasurementnoiseis0.1V.
Nowlet'sbuildourmodel:

AsIpromisedearlier,wereducedtheequationstoaverysimpleform.
Aboveall,wehavea1dimensionalsignalproblem,soeveryentityinourmodelisanumericalvalue,notamatrix.
Wehavenosuchcontrolsignaluk ,andit'soutofthegame
Asthesignalisaconstantvalue,theconstantAisjust1,becausewealreadyknowthatthenextvaluewillbesameasthepreviousone.Weareluckythatwehave
aconstantvalueinthisexample,butevenifitwereanyotherlinearnature,againwecouldeasilyassumethatthevalueAwillbe1.
ThevalueH=1,becauseweknowthatthemeasurementiscomposedofthestatevalueandsomenoise.You'llrarelyencounterreallifecasesthatHisdifferent
from1.

http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

3/5

1/6/2015

Bilgin'sBlog|KalmanFilterforDummies

Andfinally,let'sassumethatwehavethefollowingmeasurementvalues:

TIME
(ms)
VALUE
(V)

10

0.39 0.50 0.48 0.29 0.25 0.32 0.34 0.48 0.41 0.45

OK,weshouldstartfromsomewhere,suchask=0.Weshouldfindorassumesomeinitialstate.Here,wethrowoutsomeinitialvalues.Let'sassumeestimateofX0=
0,andP0=1.Thenwhydidn'twechooseP0=0forexample?It'ssimple.Ifwechosethatway,thiswouldmeanthatthere'snonoiseintheenvironment,andthis
assumptionwouldleadalltheconsequent

tobezero(remainingastheinitialstate).SowechooseP0somethingotherthatzero.

Let'swritetheTimeUpdateandMeasurementUpdateequations.

Now,let'scalculatethe

TimeUpdate

MeasurementUpdate

(prediction)

(correction)

valuesforeachiteration.

Time

Update

10

0.390

0.500

0.480

0.290

0.250

0.320

0.340

0.480

0.410

0.450

0.355

0.424

0.442

0.405

0.375

0.365

0.362

0.377

0.380

0.091

0.048

0.032

0.024

0.020

0.016

0.014

0.012

0.011

=0

=0.355

=1

=0.091

=1/(10.1)

=0.091/(0.0910.1)

=0.909

=0.476

=00.909(0.3900)

=0.3550.476(0.5000.355)

=0.35

Measurement

=0.424

Update
=(10.909).1

=(10.476).0.091

=0.091

=0.048

0.355

0.424 0.442 0.405 0.375 0.365 0.362 0.377 0.380 0.387

0.091

0.048

0.032

0.024

0.020

0.016

0.014

0.012

0.011

0.010

Here,Idisplayedthefirst2stateiterationsindetail,theothersfollowthesamepattern.I'vecompletedthe
othernumericalvaluesviaacomputeralgorithm,whichistheappropriatesolution.Ifyoutrytowriteitas
analgorithm,you'lldiscoverthatKalmanFilterisveryeasytoimplement.
Thecharthere(right)showsthattheKalmanFilteralgorithmconvergestothetruevoltagevalue.Here,I
displayedthefirst10iterationsandweclearlyseethesignsofconvergence.In50orsoiterations,it'll
convergeevenbetter.
Toenabletheconvergenceinfewersteps,youshould
Modelthesystemmoreelegantly

TheKalmanFilteralgorithmconvergestothetruthoverafew

Estimatethenoisemoreprecisely

iterations

OK.We'redone.Theonlythingtodoiscollectingthe

valueswe'vecalculated.That'sit!

References
[1]GregWelch,GaryBishop,"AnIntroductiontotheKalmanFilter",UniversityofNorthCarolinaatChapelHillDepartmentofComputerScience,2001
[2]M.S.Grewal,A.P.Andrews,"KalmanFilteringTheoryandPracticeUsingMATLAB",Wiley,2001

About|AffiliateProgram|Sitemap|FAQ's

http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

PrivacyStatement|TermsOf
Use

4/5

1/6/2015

Bilgin'sBlog|KalmanFilterforDummies

eptaxia|GameDevelopmentTwoDifferentPlatforms|VolcanoIsland|GamingThroughMyLife|AsaManThinketh|KalmanFilterforDummies|SomeTechno
Tips|MyPhDJournal

http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

Copyright2009byBilginEsme

5/5