You are on page 1of 7

ARIMAtotherescue 1 SpiderFinancialCorp,2012

ARIMAtotherescue
ThisisthesecondentryinourseriesofUnpluggedtutorials,inwhichwedelveintothedetailsofeach
ofthetimeseriesmodelswithwhichyouarealreadyfamiliar,highlightingtheunderlyingassumptions
anddrivinghometheintuitionsbehindthem.
Infinancialtimeseriesandotherfields,weoftenfaceanonstationarytimeseries,forexampletraded
security(e.g.stock,bond,commodity,etc.)pricelevels.Inthiscase,thetimeseriesexhibitseither
trending,seasonalityormerelymisguided(random)walk.Unfortunatelythebulkoftimeseriesand
econometricmethodscanbeappliedonlytostationaryprocesses,sohowdowehandlethisscenario?
Inthisissue,wetackletheARIMAmodelanextensionoftheARMAmodel,buttheARIMAmodel
appliestononstationarytimeseriesthekindoftimeserieswithoneormoreunitroots(integrated).
Onceagain,wewillstartherewiththeARIMAprocessdefinition,statingtheinputs,outputs,
parameters,stabilityconstraints,andassumptions.Thenwewillintroducetheintegrationoperatorand
drawafewguidelinesforthemodelingprocess.
Background
Anonstationarytimeseriesoftenexhibitsafewcommonpatternsincludingtrendovertime,
seasonality,andmisguidedrandomwalk.Thetrendorseasonalitycanalsobeclassifiedaseither
deterministic(functionoftime)orstochastic(functionofpastvalues).
Forstochastictrendand/orseasonality,weoftendifference(i.e.computethechangeof)theoriginal
timeseriestoinduceastationaryserieswhichcanbefurthermodeledbyanARMAtypeofprocess.
Bydefinition,theautoregressiveintegratedmovingaverage(ARIMA)processisanARMAprocessfor
thedifferencedtimeseries.
Alternatively,inasimpleformulation,anARIMA(p,d,q)isdefinedasfollow:
1 1
(1 )(1 ) (1 )
p q
i d j
i t j t
i j
L L Y L a | u
= =
= + +

OR
1 1
1 1
(1 ) (1 )
(1 )( ) (1 )
(1 ) (1 ) ... (1 ) (1 )
p q
i j
i t j t
i j
p q
i j
i t j t
i j
d d
t t t
L Z L a
L Z L a
Z Y L L L L Y
| u
| u
= =
= =
= + +
= +
= A = =


ARIMAtotherescue 2 SpiderFinancialCorp,2012

Where
-
t
Y istheobservedoutputattimet
-
d
A isthedifferenceoperatoroforder d
-
t
Z isthedifferencedtimeseriesattimet
-
t
a istheinnovation,shockorerrortermattimet
- { }
t
a timeseriesobservations:
o Areindependentandidenticallydistributed
o FollowaGaussiandistribution(i.e.
2
(0, ) o u ).
Assumptions
Lookingcloserattheformulation,weseethattheARIMAprocessisessentiallyanARMAprocessforthe
differencedtimeseriesasidefromthedifferenceoperator(
d
A ).ThesameassumptionforanARMA
processapplieshereaswell:
- TheARMAprocessgeneratesastationarytimeseries .
t
Z
- Theresiduals { }
t
a followastableGaussiandistribution.
- Thecomponentsparameter
1 2 1 2
{ , ,..., , , ,..., }
p q
| | | u u u valuesareconstants.
- Theparameter
1 2 1 2
{ , ,..., , , ,..., }
p q
| | | u u u valuesyieldastationaryprocess.
Soundsimple?Itis!AcarefulselectionoftheARMAmodelparameterscanguaranteeastationary
processforthedifferencedtimeseries(
t
Z ),buthowdoweinterprettheforecastof
t
Y using
t
Z
Integration(undifference)Operator
Inmanycases,weoftenapplyadifferenceoperatortoyieldastationarytimeseriesthatcanbeeasily
modeledusingARMAtypeofmodel.Buthowdogobacktotheoriginalundifferencedtimeseries
spaceandinterprettheARMAresults(e.g.forecast)?OurbestbetistousetheIntegrationOperator.
DEFINITION:astochastictimeseries{ }
t
Y issaidtobeintegratedoforder(d)(i.e. ~ ( )
t
Y I d ),ifthed
timesdifferencedtimeseriesyieldsaninvertibleARMArepresentation.

1 1
1
1 1
1 1
(1 )
1 1
p p
i i
i i
d k i i
t t t k t q q
j j k
j j
j j
L L
a Y Z L Z
L L
| |
t
u u

= =
=
= =

= A = = +
+ +

ARIMAtotherescue 3 SpiderFinancialCorp,2012

AND,
1
k
k
t

=
<


Now,torecover
t
Y fromthe (1 )
d
t
L Y ,weapplytheundifference(integration)operator.
Afirstorderintegrationcanbeexpressedas

2 3
0
0
1
(1 ... ...)
1
n i t
t t t
i
t t i
i
n
T n T T i
i
Z
Y Z L L L L Z L
L
Y Z
Y Y Z

=
+ +
=
= = + + + + + + =

=
= +

Forhigherorder(i.e. d order)integration,wesimplyintegratemultipletimes:
0 0 0
1 1 1
... ...
(1 ) 1 1 1
i i i t
t t t d
i i i
Z
Y Z Z L L L
L L L L

= = =
= = =



Forinstance,for 2 d = ,theintegrationoperatorisdefinedasfollow:
( )
( )
( )
2
2
2
2 3
0
1
1
1
1 1
1 ...
(1 ) 1 1
(1 2 3 4 ... ( 1) ...) ( 1)
( 1 )
t
t t t
n i
t t t
i
n
T n T T T n i
i
T T T
Z
Y Z Z L L
L L L
Y Z L L L n L Z i L
Y Y n W n i Z
W Y Y

+ +
=

= = = + + +

= + + + + + + + = +
= + + +
=


For 3 d = ,theintegrationoperatorisdefinedasfollow:

( )
3
2
3
2
0
1
1
1
1 1 2
1 1 1
1 ...
(1 ) 1 1 1
( 1)( 2) ( 1)( 2)
(1 3 6 ... ..)
2 2
( 1) ( 1 )( 2 )
2 2
2
t
t t t
n i
t t t
i
n
T n T T T T n i
i
T T T
T T T T T T
Z
Y Z Z L L
L L L L
n n i i
Y Z L L L Z L
n n n i n i
Y Y nW V Z
W Y Y
V W W Y Y Y

+ +
=


= = = + + +

+ + + + | |
= + + + + + =
|
\ .
+ + | |
= + + +
|
\ .
=
= = +

ARIMAtotherescue 4 SpiderFinancialCorp,2012

Since{ }
t
Y isanintegratedtimerseriesoforder d ,then
t
Z isastationarytimeserieswhichwecan
expressinaMArepresentation:

0 0 0 0 0 0
...
k i i i k i
t t k t k i
k i i i k i
Y a L L L L a L L ,

= = = = = =
= =


Wecancomputetheconditionalvarianceattime T n + giventheinformationavailableattimet :
( )
1
2 2
1 1
0 0 1
0
( | ... ) ( )
Where:
1
n
i k
T n T T t i k a i
i k i
i
i i k k
k
o o
Var Y Y Y Y Var a L L , o
,
,

+
= = =

=
= =
=
= =


IMPORTANT:NumXLhasafunctionINTG()thatcomputestheintegralofaseasonaldifferenced(i.e.
(1 )
d s d
t s t t
Z Y L Y = A = )timeseries.Torecoveradifferencedtimeseriesoforderd,set 1 s = andpass
ontheinitialconditions(e.g.
1
, ,...,
T T T d
Y Y Y

),anditwillrecovertheoriginaldataseries.

ARIMAtotherescue 5 SpiderFinancialCorp,2012

ARIMAMachine
TheARIMAprocessisasimplemachinethatretainslimitedinformationaboutitspastdifferenced
outputsandtheshocksithasexperienced.Inamoresystematicview,theARIMAprocessormachine
canbeviewedasbelow:

NotethatweareobservingtheintegratedoutputoftheARMAprocess(
t
Y ),butthemachineprocesses
thedifferencedoutputs(
t
Z ).TheINTGblockreferencestheintegrationoperator.
Howdoweknowifwehaveaunitrootinourtimeseries?
Asidefromthestatisticaltestsforunitroot(e.g.ADF,KPSS,etc.),thereareafewvisualcluesfor
detectingunitrootusingtheACFandPACFplots.Forinstance,atimeserieswithunitrootwillexhibit
highandveryslowdecayingACFvaluesforalllags.OnthePACFplot,thePACFvalueforthefirstlagis
almostone(1),andthePACFvaluesforlagordergreaterthanoneareinsignificant.
Forstatisticaltesting,theAugmentedDickeyFuller(ADF)testwillexaminetheevidenceforaunit
root,eveninthepresenceofdeterministictrendorsquaredtimetrend.
Note:Startingin1.55(LYNX),NumXLnativelysupportstheADFtestwithastepdownoptimization
procedure.
StatisticalCharacteristics
InourdescriptionoftheARIMAprocess,wehighlightedasingleinputstimulus:shocks/innovations,
emphasizinghowtheypropagatethroughouttheARIMAmachinerytogeneratetheobservedoutput.

ARIMAtotherescue 6 SpiderFinancialCorp,2012

TheARIMAmachineisbasicallyanARMAmachine,buttheoutputisintegratedbeforewecanobserve
it.Howdoesthisaffecttheoutputdistribution?
Whydowecare?
Thestatisticaldistribution(i.e. )oftheoutput(
T n
Y
+
)ispivotalforconductingaforecastand/or
establishingaconfidenceintervalatanyfuturetime( n).

2
/ 2 / 2
~ ( , )

T n T n T n
T n l T n T n T n u T n
Y
Z Y Z
o o
o
o o
+ + +
+ + + + +
s s +

Where
-

T n
Y
+
istheoutofsampleforecastattime T n +
-
/ 2
l
Z
o
isthelowercriticalvaluefor / 2 o significancelevel
-
/ 2
u
Z
o
istheuppercriticalvaluefor / 2 o significancelevel
-
2
T n
o
+
istheconditionalvarianceattime T n +
Bynow,theimportanceofunderstandingtheoutputstatisticaldistributionshouldbeclear.Nowhow
dowegoaboutformingthatunderstanding?
Backtothedefinition,thedifferencedtimeseries { }
t
Z ismodeledasastationaryARMAprocess.Lets
convertittoaninfiniteorderMAmodel:
1 1
1
1 0
1
0
(1 ) (1 )
1
(1 )
1
1
p q
i j
i T j t
i j
q
j
j
j k k
T t k t t k p
i k k
i
i
L Z L a
L
Z a L a a L
L
| u
u

|

= =

=
= =
=
= +
+
= = + =


Now,letsrecovertheoriginaltimeseriesfrom { }
t
Z
Example1:Letsconsiderthefollowingdifferencedseries (1 )
t t
Z L Y = .Torecoverthe{ }
t
Y time
series,wesimplyaddupallthedifferencestodate.

1
1
1 1 1 0
T T T
n n n i
j
T n T T i T j T i T T i j
i i j o i j
Z Y Y
Y Y Z Y L a Y a


+ + + +
= = = = =
=
| | | |
= + = + = +
| |
\ . \ .

ARIMAtotherescue 7 SpiderFinancialCorp,2012

Now,thevarianceoftheforecastisexpressedasfollow:

2
2
1 0
( )
n n k
T n a i
k i
Var Y o

+
= =
| |
=
|
\ .


Aswesee,althoughcomputingtheforecastissimpleexerciseofsummingallpriordifferences,the
variancecalculationismuchmoreinvolved.
Furthermore,as 1 n ,the
1
1
T n p
i
i
Z

|
+
=

,sothe
T n
Y
+
estimate/forecastasymptoticallyapproaches
thedeterministiclineartrenddefinedby:
1
1
T n T p
i
i
n
Y Y

|
+
=

.
Note:Forhigherorderintegration(d>1),itcanbeeasilyshownthatlongrunforecastvaluesofthetime
seriesvalueswouldasymptoticallyfollowapolynomialofthesameorder.
Conclusion
Insimpleterms,anARIMAprocessismerelyanARMAprocesswhoseoutputshavegonethroughan
integrator.Theintegratorcausestheobservedtimeseries

{ }
T
Y tobenonstationary.Theintegration
processintroducestheunitrootinto{ }
T
Y .Integratingmultipletimesintroducesmultipleunitrootsinto
theoutputtimeseries.ThisiswhythewordintegratedisusedinARIMA.
Themaintakeawayofthispaperisthatdifferencingisaspecialtransformationprocedurethatisaimed
toconvertanonstationarytimeseriesintoastationaryone.Likealltransformations,caremustbe
takenwhenweinterprettheresultsbackintotheoriginaltimeseriesspace.
Noticethattheunitrootmodeling(e.g.ARIMA)isintendedtocaptureastochastictrendanditisnot
suitedforadeterministictrend.Ifyoususpectthepresenceofadeterministictrend,youshouldexplore
thisavenuefirst(i.e.regressovertime).Atthatpoint,youmaychoosetotaketheresidualsandapply
anARMAtypeofprocesstoexploitanyremainingdynamics.

You might also like