ARMAUnplugged 1 SpiderFinancialCorp,2012

ARMAUnplugged
ThisisthefirstentryinourseriesofUnpluggedtutorials,inwhichwedelveintothedetailsofeachof
thetimeseriesmodelswithwhichyouarealreadyfamiliar,highlightingtheunderlyingassumptionsand
drivinghometheintuitionsbehindthem.
Inthisissue,wetackletheARMAmodelacornerstoneintimeseriesmodeling.Unlikeearlieranalysis
issues,wewillstartherewiththeARMAprocessdefinition,statetheinputs,outputs,parameters,
stabilityconstraints,assumptions,andfinallydrawafewguidelinesforthemodelingprocess.
Background
Bydefinition,theautoregressivemovingaverage(ARMA)isastationarystochasticprocessmadeupof
sumsofautoregressiveandmovingaveragecomponents.
Alternatively,inasimpleformulation:

1 1 2 2 1 1 2 2
1 1 2 2 1 1 2 2
... ...
( ... ) ( ... )
t t t p t p t t t q t q
t t t p t p t t q t q t
y y y y a a a a
OR
y y y y a a a a
| | | u u u
| | | u u u


= + + + + +
= + + + + + + + + +

Where
1.
t
y istheobservedoutputattimet.
2.
t
a istheinnovation,shockorerrortermattimet.
3. { }
t
a timeseriesobservations:
a. Areindependentandidenticallydistributed( ~ . .
t
a i i d )
b. FollowaGaussiandistribution(i.e.
2
(0, ) o u ).
Note:Thevarianceoftheshocksdistribution(i.e.
2
o )istimeinvariant.
Usingbackshiftnotations(i.e. L ),wecanexpresstheARMAprocessasfollows:

2 2
1 2 1 2
(1 ... )( ) (1 ... )
p q
p t q t
L L L y L L L a | | | u u u = + + + +
Assumptions
Letslookcloserattheformulation.TheARMAprocessissimplyaweightedsumofthepastoutput
observationsandshocks,withfewkeyassumptions:

ARMAUnplugged 2 SpiderFinancialCorp,2012

1. TheARMAprocessgeneratesastationarytimeseries({ }
t
y ).
2. Theresiduals { }
t
a followastableGaussiandistribution.
3. Thecomponentsparameter
1 2 1 2
{ , ,..., , , ,..., }
p q
| | | u u u valuesareconstants.
4. Theparameter
1 2 1 2
{ , ,..., , , ,..., }
p q
| | | u u u valuesyieldastationaryARMAprocess.
Whatdotheseassumptionsmean?
Astochasticprocessisacounterpartofadeterministicprocess;itdescribestheevolutionofarandom
variableovertime.Inourcase,therandomvariableis
t
y .
Next,arethe{ }
t
y valuesindependent?Aretheyidenticallydistributed?Ifso,
t
y shouldnotbedescribed
byastochasticprocess,butinsteadbyaprobabilisticdistributionmodel.
Forcaseswhere { }
t
y valuesarenotindependent(e.g.
t
y valueispathdependent),astochasticmodel
similartoARMAisinordertocapturetheevolutionof
t
y .
TheARMAprocessonlycapturestheserialcorrelation(i.e.autocorrelation)betweentheobservations.
Inplainwords,theARMAprocesssumsupthevaluesofpastobservations,nottheirsquaredvaluesor
theirlogarithms,etc.Higherorderdependencymandatesadifferentprocess(e.g.ARCH/GARCH,non
linearmodels,etc.).
Therearenumerousexamplesofastochasticprocesswherepastvaluesaffectcurrentones.For
instance,inasalesofficethatreceivesRFQsonanongoingbasis,somearerealizedassaleswon,some
assaleslost,andafewspilledoverintothenextmonth.Asaresult,inanygivenmonth,someofthe
saleswoncasesoriginateasRFQsorarerepeatsalesfromthepriormonths.
Whataretheshocks,innovationsorerrorterms?
Thisisdifficultquestion,andtheanswerisnolessconfusing.Still,letsgiveitatry:Insimplewords,the
errorterminagivenmodelisacatchallbucketforallthevariationsthatthemodeldoesnotexplain.
Confused?Letsputitadifferentway.Inanygivensystem,therearepossiblytensofvariablesthat
affecttheevolutionof
t
y ,butthemodelcapturesfewofthemandwillbundletherestasanerrorterm
initsformula(i.e.
t
a ).
Stilllost?Letsuseanexample.Forastockpriceprocess,therearepossiblyhundredsoffactorsthat
drivethepricelevelup/down,including:
DividendsandSplitannouncements
Quarterlyearningsreports
Mergerandacquisition(M&A)activities
Legalevents,e.g.thethreatofclassactionlawsuits.

ARMAUnplugged 3 SpiderFinancialCorp,2012

Others
Amodel,bydesign,isasimplificationofacomplexreality,sowhateverweleaveoutsidethemodelis
automaticallybundledintheerrorterm.
TheARMAprocessassumesthatthecollectiveeffectofallthosefactorsactsmoreorlesslikeGaussian
noise.
Whydowecareaboutpastshocks?
Unlikearegressionmodel,theoccurrenceofastimulus(e.g.shock)mayhaveaneffectonthecurrent
level,andpossiblyfuturelevels.Forinstance,acorporateevent(e.g.M&Aactivity)affectstheunderling
companysstockprice,butthechangemaytakesometimetohaveitsfullimpact,asmarket
participantsabsorb/analyzetheavailableinformationandreactaccordingly.
Thisbegsthequestion:dontthepastvaluesoftheoutputalreadyhavetheshockspastinformation?
YES,theshockshistoryisalreadyaccountedforinthepastoutputlevels.AnARMAmodelcanbesolely
representedasapureautoregressive(AR)model,butthestoragerequirementofsuchasystemin
infinite.ThisisthesolereasontoincludetheMAcomponent:tosaveonstorageandsimplifythe
formulation.

2 2
1 2 1 2
2
1 2
2
1 2
2 3
1 2 3
(1 ... )( ) (1 ... )
(1 ... )
( )
(1 ... )
(1 ... ...)( )
p q
p t q t
p
p
t t q
q
N
N t t
L L L y L L L a
L L L
y a
L L L
L L L L y a
| | | u u u
| | |

u u u

= + + + +

=
+ + + +
=

ARMAUnplugged 4 SpiderFinancialCorp,2012

ARMAMachine
TheARMAprocessisasimplemachinethatretainslimitedinformationaboutitspastoutputsandthe
shocksithasexperienced.Inamoresystematicview,theARMAprocessormachinecanbeviewedas
below:

Inessence,thephysicalimplementationofanARMA(P,Q)processrequiresP+Qstorages,orthememory
requirementsofanARMA(P,Q)arefinite(P+Q).
Attimezero(0),theARMAmachineisresetandallitsstorages(i.e.
1 2 1 2
{ , ,..., , , ,..., }
y y y a a a
p q
S S S S S S )are
settozero.Asnewshocks(i.e.
t
a )begintoarriveinoursystem,theinternalstoragesgetupdatedwith
thenewobservedoutput(
t
y )andrealizedshocks.
Furthermore,theARcomponentintheARMArepresentsapositivefeedback(addingtheweightedsum
ofthepastoutput),andthismaycausetheoutputtobenonstationary.Thefeedbackeffectforthe
shocksislessofaconcern,astheshockshavezeromeanandareindependent.
Forastable(i.e.stationary)ARMAprocess,therootsofthecharacteristicARequationmustbewithina
unitcircle.

2
1 2 1 2
1 ... (1 )(1 )...(1 )
1
p
p p
k
L L L L L L | | |

=
<

Where
1.
k
isthekthroot
2. 1 k p s s
Note:intheeventthattheMAandARcomponentshaveanycommonroot,theARMAorders(i.e.Pand
Q)shouldbereduced.

ARMAUnplugged 5 SpiderFinancialCorp,2012

StationaryARMA
Nowthatwehaveastationary(stable)ARMAprocess,letsshiftgearsandexaminetheoutputseries
characteristics.

1 1 2 2 1 1 2 2
... ...
t t t p t p t t t q t q
y y y y a a a a | | | u u u

= + + + + +
Marginalmean(longrunmean)

1 1 2 2 1 1 2 2
1 1 2 2
1 2
[ ] [ ... ... ]
[ ] [ ] [ ] ... [ ]
[ ]
1 ..
t t t p t p t t t q t q
t t t p t p
t
p
E y E y y y a a a a
E y E y E y E y
E y
| | | u u u
| | |

| | |


= + + + + + + + +
= + + + +
=

ForastationaryARMAprocess,the
1
1
p
k
k
|
=
=

.
Marginalvariance(longrunvariance)
TheformulaforlongtermvarianceofanARMAmodelismoreinvolved:tosolveforit,IderivetheMA
representationoftheARMAprocess,afterwhichItakethevariance,sincealltermsareindependent.
Forillustration,letsconsidertheARMA(1,1)process:

2 2 3 1
1
1 2
2 2 2 2 2 2 2
(1 )( ) (1 )
1
(1 ( ) ( ) ( ) ... ( ) ...)
1
( ) ( ) ... ( ) ...
[ ] [ ] (1 ( ) ( ) ... ( ) ...)
t t
N N
t t t
N
t t t t t N
N
t t a
L y L a
L
y a L L L L a
L
y a a a a
Var y Var y
Va
| u
u
| u | | u | | u | | u
|
| u | | u | | u
| u | | u | | u o

= +
+
= = + + + + + + + + + +

= + + + + + + + +
= = + + + + + + + +
2
2 2 4 2 2 2 2
2
2
2 2
2
( )
[ ] (1 ( ) (1 ... ...)) (1 )
1
1 2
( )
1
N
t a a
y a
r y
| u
| u | | | o o
|
u| u
o o
|

+
= + + + + + + + = +

+ +
=

Again,theARMAprocessmustbestationaryforthemarginal(unconditional)variancetoexist.
Note:Inmydiscussionabove,Iamnotmakingadistinctionbetweenmerelytheabsenceofaunitroot
inthecharacteristicequationandthestationarityoftheprocess.Theyarerelated,buttheabsenceofa
unitrootisnotaguaranteeofstationarity.Still,theunitrootmustbelieinsidetheunitcircletobe
accurate.

ARMAUnplugged 6 SpiderFinancialCorp,2012

Conclusion
Letsrecapwhatwehavedonesofar.FirstweexaminedastationaryARMAprocess,alongwithits
formulation,inputs,assumptions,andstoragerequirements.Next,weshowedthatanARMAprocess
incorporatesitsoutputvalues(autocorrelation)andshocksitexperiencedearlierinthecurrentoutput.
Finally,weshowedthatthestationaryARMAprocessproducesatimeserieswithastablelongrun
meanandvariance.
Inourdataanalysis,beforeweproposeanARMAmodel,weoughttoverifythestationarityassumption
andthefinitememoryrequirements.
1. Intheeventthedataseriesexhibitsadeterministictrend,weneedtoremove(detrend)itfirst,
andthenusetheresidualsforARMA.
2. Intheeventthedatasetexhibitsastochastictrend(e.g.randomwalk)orseasonality,weneed
toentertainARIMA/SARIMA.
3. Finally,thecorrelogram(i.e.ACF/PACF)canbeusedtogaugethememoryrequirementofthe
model;weshouldexpecteitherACForPACFtodecayquicklyafterafewlags.Ifnot,thiscanbe
asignofnonstationarityoralongtermpattern(e.g.ARFIMA).