Applied Econometrics

Introduction to Time Series

Roman Horvth
Lecture 3

Contents
Stationarity
What it is and what it is for

Some basic time series models

Autoregressive (AR)
Moving average (MA)

Consequences of non-stationarity (spurious

regression)
Testing for (non)-stationarity
Dickey-Fuller test
Augmented Dickey-Fuller test
2

(Weak) Stationarity
Xt is stationary if:
the series fluctuates around a constant long run

mean
Xt has finite variance which is not dependent upon time
Covariance between two values of Xt depends only on the
difference apart in time (e.g. covariance between Xt and Xt-1 is
the same as for Xt-8 and Xt-9)
E(Xt) =
(mean is constant in t)
Var(Xt) = 2
(variance is constant in t)
Cov(Xt ,Xt+k) = (k) (covariance is constant in t)
If data not stationary, spurious regression problem

Examples of Times Series Models

AR autoregressive models
Xt = + *Xt-1 + ut is called AR(1) process
Xt = + 1*Xt-1 + 2*Xt-2 +.+k*Xt-k + ut is AR(k) process

MA moving average models

Xt = + ut + 2*ut-1 is called MA(1) process
Xt = + ut + 2*ut-1 +.+k*ut-k .is called MA(k) process

If you combine AR and MA process, you get

ARMA process
E.g. ARMA (1,1) is Xt = +*Xt-1+ut+ 2 ut-1
4

Is MA and AR process stationary?

Compute mean, variance and covariance and
check if it depends on time
For AR process, you may easily derive that the
process is stationary if < 1
For MA (1) process, mean is , variance is ut2
*(1+22) and covariance cov(Xt , Xt-k) is either 0
if k>1 or ut2*2 ,so it does not depend on time
(MA(k) is stationary process)
5

Example of Stationary Time Series

White noise process:

Xt = ut

ut ~ IID(0, 2 )

0.6
0.4
0.2
0
-0.2
-0.4
-0.6

Another Example of Stationary Time Series

Xt = 0.5*Xt-1 + ut

ut ~ IID(0, 2 )

Stationary without drift

0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
7

Example of Non-stationary Time Series

Yt = + *t + ut , where t is time trend
Take the expected value E(Yt ) = + *t ,
clearly the mean depends on time and the series
is non-stationary

Non-stationary time series

In contrast a non-stationary time series has at least
one of the following characteristics:
Does not have a long run mean which the series
returns
Variance is dependent upon time and goes to
infinity as the sample period approaches infinity
Correlogram does not die out - long memory

Example of Non-stationary Time Series

2003 Q1

2002 Q2

2001 Q3

2000 Q4

2000 Q1

1999 Q2

1998 Q3

1997 Q4

1997 Q1

1996 Q2

1995 Q3

1994 Q4

1994 Q1

1993 Q2

1992 Q3

120
100
80
60
40
20
0

UK GDP Level

The level of GDP is not constant; the mean increases over

10
time.

Non-stationary time series correlogram

UK GDP (Yt)

1.0
ACF-Y
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

For non-stationary series the Autocorrelation Function (ACF)

declines towards zero at a slow rate as k increases.
11

Possible solutions of non-stationarity

2003 Q1

2002 Q2

2001 Q3

2000 Q4

2000 Q1

1999 Q2

1998 Q3

1997 Q4

1997 Q1

1996 Q2

1995 Q3

1994 Q4

1994 Q1

1993 Q2

1.4
1.2
1
0.8
0.6
0.4
0.2
0

1992 Q3

Some transformation = first differenece, logarithm, second

difference ...
First difference of UK GDP (Yt = Yt - Yt-1) is stationary:
- growth rate is reasonably constant through time
- variance is also reasonably constant through time

12

Stationary time series - correlogram

UK GDP Growth ( Yt)
1.00
ACF-DY
0.75

0.50

0.25

0.00

-0.25

-0.50

-0.75

ACF decline towards zero as k increases

Decline of ACF is rapid for stationary series
13

Non-stationary Time Series Continued

Random Walk
Xt = Xt-1 + ut , where ut ~ IID(0, 2 )
Mean is constant in t: E(Xt) = E(Xt-1)
X1 = X0 + u1
(take initial value X0)
X2 = X1 + u2 = (X0 + u1 ) + u2

Xt = X0 + u1 + u2 ++ ut (take expectations)
E(Xt) = E(X0 + u1 + u2 ++ ut) = E(X0) = constant
Variance is not constant in t:
Var(Xt) = Var(X0) + Var(u1) ++ Var(ut)
= 0 + 2 ++ 2 = t 2

14

Random walk
Xt = Xt-1 + ut

ut ~ IID(0, 2 )

2.5

2
1.5
1
0.5
0
15

Relationship between stationary and nonstationary process

AR(1) process: Xt = + Xt-1 + ut ;(ut ~ IID(0, 2 ))
< 1stationary process - process forgets
past
= 1non-stationary process - process does
not forget past
= 0 without drift
0 with drift

MA process is always stationary

16

Summary on basic time series processes

AR (k) process
MA(k) process
ARMA (p,l) process
If you k-th difference the data, then you have
ARIMA (p,k,l) estimation of ARIMA models
is a subject of next lecture

17

Spurious Regression
( spurious correlation)
Problem that time-series data usually includes trend
Result:
Spurious correlation (variables with similar trends
are correlated)
Spurious regression (independent variable with
similar trend looks as dependent = strong statistical
relationship)
coefficient significant (high adjusted-R2, large
t-statistics) ... even if unrelated in economic
terms
18

How to avoid spurious regression:

3 approaches to non-stationarity
1.

Include a time trend as an independent variable (oldfashioned)

yt = c + 1xt + 2t + ut .....(t = 1,2, ..., T)

2.

1st difference the data if variables I(1); 2nd difference if

I(2)
= converts non-stationary variables into stationary
variables
Problems:
theory often about levels
detrending loss of information

3.

Cointegration + ECM
= Long-run relationship + short-run adjustment
19

How do we identify non-stationary

processes?
(A) Informal methods:
- Plot time series
- Correlogram

(B) Formal methods:

- Statistical test for stationarity
- Dickey-Fuller tests.
20

Informal Procedures to identify non-stationary

processes
(a) Constant mean?
12
RW2

10

0
0

50

100

150

200

250

300

350

400

450

500

(b) Constant variance?

var

200
150
100
50
0
-50
-100
-150

21

-200
0

50

100

150

200

250

300

350

400

450

500

Informal Procedures to identify non-stationary

processes
Diagnostic test Correlogram for stationary process (dies out
rapidly, series has no memory)
0.50
whitenoise

0.25
0.00
-0.25

0
1.0

50

100

150

200

250

300

350

400

450

500

ACF-whitenoise

0.5
0.0
-0.5

10

22

Informal Procedures to identify non-stationary

processes
Diagnostic test Correlogram for a random walk (does not die out,
high autocorrelation for large values of k)
12.5
randomwalk

10.0
7.5
5.0
2.5
0.0
0
1.00

50

100

150

200

250

300

350

400

450

500

ACF-randomwalk

0.75
0.50
0.25

10

23

Dickey-Fuller Test
Test based on Yt = Yt-1 + ut
- DF test to determine whether =1
Yes unit root non-stationary
No no unit root
Dynamic model:
Subtract Yt-1 ... Yt - Yt-1 = (-1)Yt-1 + ut
Reparameterise: Yt = Yt-1 + ut
where = (-1)
Test =0 equivalent to test =1
24

Augmented Dickey-Fuller Test

Augment dynamic model Yt = Yt-1 + ut:
1) Constant or drift term (0)
Yt = 0 + Yt-1 + ut
2) Time trend (T)
Yt = 0 + T + Yt-1 + ut
3) Lagged values of the dependent variable
Yt = 0 + T + Yt-1 + 1Yt-1 + 2Yt-2 + ... + ut
Find the right specification, trade-off parsimony vs. white
noise in residual
25

Critical values
DF/ADF use t- and F-statistics but critical values are
not standard
Problems:
distributions of these statistics are non-standard
special tables of critical values (derived from
numerical simulations)
Usual t- and F-tests not valid in presence of unit
roots

26