Professional Documents
Culture Documents
Paul Cowpertwait
Preface
Textbook
The text book for this part of the course is ‘Introductory Time Series
with R’ by Cowpertwait and Metcalfe (Springer, 2009), and is available
locally in city bookstores.
Software
This course uses R, which is an open source and freely available software
environment, based on the S language that was originally developed at
the Bell (AT&T) Laboratories. The S language was designed for the
purpose of data handling and analysis, making it an attractive tool to
the applied statistician and econometrician. This, together with power-
ful graphics utilities, makes R ideal for a time series analysis. Available
libraries provide a range of functions especially suited to econometrics
and finance, making R a strong competitor to specialist financial soft-
ware; R is also supported by a number of internationally recognised
economic and finance groups and university departments (see ‘mem-
bers’ at www.r-project.org).
2
Installing R
1. Go to: http://cran.stat.auckland.ac.nz/
2. Click ‘Windows’
3. Click ‘base’
4. Click link to setup program (e.g. called R-2.11.1 for version
2.11.1 - pick the latest version)
5. Download to setup program to your machine (e.g. to Desktop)
6. Double-click the setup program that is now on your machine. This
will install R. Follow any instructions on the screen. When R is in-
stalled carry out the following instructions to install the time series
library. (You can also install any other libraries in this way; e.g.
‘fseries’ or ‘vars’ for more advanced financial time series analysis
should you need them.)
7. Go back to http://cran.stat.auckland.ac.nz/
8. Click ‘Windows’
9. Click ‘contrib’
10. Click ‘2.11.1’ (replace this with the version you are using)
3
11. Download the following (e.g. ‘save to desktop’): zoo_1.0.zip,
tseries_0.1.zip, quadprog_1.zip (use the latest versions here
also)
12. Now start R (e.g. Double click R icon)
13. Click “Packages” on the menu bar in R followed by “install from
local zip file”
14. Browse for zoo_1.zip (i.e. go to where you saved the file, e.g.
Desktop)
15. Click OK
16. Now repeat for the other two zip files: tseries_0.1.zip and
quadprog_1.zip (again using latest versions of these packages)
4
1. Notation and basic statistical models
Expectation
5
Covariance
If we have two variables (x, y), the variance may be generalised to the
covariance, γ(x, y) defined by:
γ(x, y) = E{(x − µx)(y − µy )}
6
Covariance explained
If data pairs are plotted, the lines x = x and y = y divide the plot into
quadrants.
Points in the lower left quadrant have both (xi − x) and (yi − y)
negative, so the product is positive.
In contrast, points in the upper left and lower right quadrants make a
negative contribution to the covariance.
7
Correlation
8
Stationarity
If the mean function is constant in time we say that the time series
is stationary in the mean and we can then use the whole series to
estimate the mean µ.
9
Autocorrelation
The mean and variance play an important role in the study of statistical
distributions, because they summarise two key distributional properties
– a central location and the spread.
In the study of time series models a key role is played by the second-
order properties, which includes the mean and variance, but also the
serial correlation.
Consider a time series model which is stationary in the mean and vari-
ance. The variables may be correlated, and the model is second-order
stationary if the correlation between variables depends only on the
number of time steps separating them, and the mean and variance do
not change with time.
The number of time steps between the variables is known as the lag.
10
Autocovariance function
11
Autocorrelation function
12
Wave height example
13
wave height (mm)
500
−500
Time
(a) Wave height over 39.7 seconds
wave height (mm)
600
0
−600
0 10 20 30 40 50 60
Time
(b) Wave height over 6 seconds
14
Correlogram
> acf(ts(waveht))
• The x-axis gives the lag (k) and the y-axis gives the autocorrelation
(rk ) at each lag. For the wave example, the unit of lag is the
sampling interval, 0.1 second. Correlation is dimensionless, so there
is no unit for the y-axis.
• The dotted lines on the correlogram represent the 5% significance
levels. If rk falls outside these lines we have evidence against the
null hypothesis, that ρk = 0, at the 5% level. But, we should
be careful about interpreting such multiple hypothesis tests, since
even if the null hypothesis is true we expect 5% of the estimates,
rk , to fall outside the lines.
• The lag 0 autocorrelation is always 1 and is shown on the plot.
Its inclusion, helps us compare values of the other autocorrelations
relative to the theoretical maximum of 1.
15
1.0
0.5
ACF
0.0
−0.5
0 5 10 15 20 25
Lag
16
Discussion based on air passenger data
17
1.0
0.6
ACF
0.2
−0.2
lag (month)
Figure 3: Correlogram for the air passenger data over the period 1949–1960. The gradual decay
is typical of a time series containing a trend. The peak at 1 year indicates seasonal variation.
18
White noise
A residual error is the difference between the observed value and the
model predicted value at time t.
Suppose the model is defined for the variable yt, and ŷt is the value
predicted by the model, the residual error xt is:
xt = yt − ŷt
As the residual errors occur in time, they form a time series: x1, x2, . . . , xn.
If a model has accounted for all the serial correlation in the data, the
residual series would be serially uncorrelated, so that a correlogram of
the residual series would exhibit no obvious patterns.
19
White noise: Definition
This implies that the variables all have the same variance σ 2 and
cor(wi, wj ) = 0 for all i 6= j.
20
Simulation
21
Simulation in R
> set.seed(1)
> w = rnorm(100)
> plot(w, type = "l")
22
2
1
w
0
−1
−2
0 20 40 60 80 100
time
23
Second-order properties and the correlogram
24
Simulated white noise
Simulated white noise data will not have autocorrelations that are ex-
actly zero (when k 6= 0), because of sampling variation.
> set.seed(2)
> acf(rnorm(100))
25
1.0
0.6
ACF
0.2
−0.2
0 5 10 15 20
Lag
Figure 5: Correlogram of a simulated white noise series. The underlying autocorrelations are
all zero (except at lag 0); the statistically significant value at lag 7 is due to sampling variation.
26
Random walks
Substituting xt−1 = xt−2 + wt−1 and then for xt−2, followed by xt−3
and so on (a process known as ‘back substitution’) gives:
xt = wt + wt−1 + wt−2 + . . .
In practice, the series will not be infinite but will start at some time
t = 1. Hence,
xt = w1 + w2 + . . . + wt
27
The backward shift operator
28
The difference operator
29
Simulation
> x = w = rnorm(1000)
> for (t in 2:1000) x[t] = x[t - 1] + w[t]
> plot(x, type = "l")
The correspondence between the R code above and the equation for a
random walk should be noted.
30
80
60
40
x
20
0
Index
Figure 6: Time plot of a simulated random walk. The series exhibits an increasing trend.
However, this is purely stochastic and due to the high serial correlation.
0.0 0.2 0.4 0.6 0.8 1.0
ACF
0 5 10 15 20 25 30
Lag
Figure 7: The correlogram for the simulated random walk. A gradual decay from a high serial
correlation is a notable feature of a random walk series.
31
Diagnostics: Simulated random walk series
The first order differences of a random walk are a white noise series,
so the correlogram of the series of differences can be used to assess
whether a given series is reasonably modelled as a random walk.
> acf(diff(x))
32
0.0 0.2 0.4 0.6 0.8 1.0
ACF
0 5 10 15 20 25 30
Lag
Figure 8: Correlogram of differenced series. If a series follows a random walk the differenced
series will be white noise.
33
Correlogram of differenced exchange rates
Financial series often behave like random walks. Recall the exchange
rate series (NZ dollars per UK pounds). The data are close to a random
walk (although there is a significant value in the correlogram at lag 1):
34
1.0
0.6
ACF
0.2
−0.2
0 1 2 3
Lag
Figure 9: Correlogram of first-order differences of the exchange rate series (UK pounds to NZ
dollars: 1991-2000). The significant value at lag 1 indicates that an extension of the random
walk model is needed for this series.
35
Random walk with drift
[1] 0.03987
> t.test(DP)
data: DP
t = 2.2467, df = 670, p-value = 0.02498
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
0.005025729 0.074706015
sample estimates:
mean of x
0.03986587
36
45
40
Closing Price
35
30
25
20
Day
2
1
0
−1
−2
Day
37
Series DP
0.0 0.2 0.4 0.6 0.8 1.0
ACF
0 5 10 15 20 25
Lag
Figure 12: Acf of Lag 1 differences of daily closing prices of Hewlett-Packard stock.
38
2. Models for stationary series
Stationary series
Recall that a time series is stationary if the mean, variance, and auto-
correlation do not change with time.
The models in this chapter may be used for stationary residual series
or other stationary series.
39
Autoregressive models
40
Notes
41
Stationary and non-stationary AR processes
42
Examples
43
Correlogram of an AR(1) process
44
Partial autocorrelation
45
Simulation
> set.seed(1)
> x = w = rnorm(100)
> for (t in 2:100) x[t] = 0.7 * x[t - 1] + w[t]
> plot(x)
> acf(x)
> pacf(x)
46
2
x
0 20 40 60 80 100
index
(a) Time plot.
1.0
0.5
ACF
0.0
0 5 10 15 20
lag
(b) Correlogram.
0.5
Partial ACF
0.0
5 10 15 20
lag
(c) Partial Correlogram.
Figure 13: A simulated AR(1) process: xt = 0.7xt−1 + wt . Note that in the partial-correlogram
(c) only the first lag is significant, which is usually the case when the underlying process is
AR(1).
47
Fitted models
[1] 1
> x.ar$ar
[1] 0.601
48
AIC
In the function ar, the model with the smallest AIC is selected as the
best fitting AR model.
49
Exchange rate series: Fitted AR model
An AR(1) model is fitted to the exchange rate series and the upper
bound of the confidence interval for the parameter includes 1, consis-
tent with the earlier conclusion that a random walk provides a good
approximation for this series.
[1] 2.82
> Z.ar$order
[1] 1
> Z.ar$ar
[1] 0.89
> acf(Z.ar$res[-1])
50
1.0
0.6
ACF
0.2
−0.2
0 5 10 15
Lag
Figure 14: The correlogram of residual series for the AR(1) model fitted to the exchange rate
data.
51
Moving average models
52
Fitted MA models
> set.seed(1)
> b <- c(0.8, 0.6, 0.4)
> x <- w <- rnorm(1000)
> for (t in 4:1000) for (j in 1:3) x[t] <- x[t] + b[j] * w[t -
j]
> plot(x, type = "l")
> acf(x)
> x.ma <- arima(x, order = c(0, 0, 3))
> x.ma
Call:
arima(x = x, order = c(0, 0, 3))
Coefficients:
ma1 ma2 ma3 intercept
0.790 0.566 0.396 -0.032
s.e. 0.031 0.035 0.032 0.090
53
Exchange rate series: Fitted MA model
In the code below, an MA(1) model is fitted to the exchange rate series.
Call:
arima(x = x.ts, order = c(0, 0, 1))
Coefficients:
ma1 intercept
1.000 2.833
s.e. 0.072 0.065
> acf(x.ma$res[-1])
54
1.0
0.5
ACF
0.0
−0.5
0 5 10 15
Lag
Figure 15: The correlogram of residual series for the MA(1) model fitted to the exchange rate
data.
55
Mixed models: The ARMA process
56
The following points should be noted about an ARMA(p, q) process:
(a) The process is stationary when the roots of θ all exceed unity in
absolute value.
(b) The AR(p) model is the special case ARMA(p, 0).
(c) The MA(q) model is the special case ARMA(0, q).
(d) Parameter parsimony. When fitting to data, an ARMA model
will often be more parameter efficient (i.e. require fewer parame-
ters) than a single MA or AR model.
(e) Parameter redundany. When θ and φ share a common factor,
the model can be simplified. For example, the model (1 − 12 B)(1 −
1 1 1
3 B)xt = (1 − 2 B)at can be written (1 − 3 B)xt = at .
57
Simulation and fitting
The ARMA process, and the more general ARIMA processes discussed
in the next section, can be simulated using the R function arima.sim.
> set.seed(1)
> x <- arima.sim(n = 10000, list(ar = -0.6, ma = 0.5))
> coef(arima(x, order = c(1, 0, 1)))
58
Exchange rate series
[1] -3.53
> AIC(x.ar)
[1] -37.4
> AIC(x.arma)
[1] -42.3
> x.arma
Call:
arima(x = x.ts, order = c(1, 0, 1))
Coefficients:
ar1 ma1 intercept
0.892 0.532 2.960
s.e. 0.076 0.202 0.244
59
1.0
0.6
ACF
0.2
−0.2
0 1 2 3
Lag
Figure 16: The correlogram of residual series for the ARMA(1,1) model fitted to the exchange
rate data.
60
3. Non-stationary models
Differencing a series {xt} can remove trends, whether these trends are
stochastic, as in a random walk, or deterministic, as in the case of a
linear trend.
In the case of a random walk, xt = xt−1 + wt, the first order differenced
series is white noise {wt}, i.e. ∇xt = xt −xt−1 = wt, and so stationary.
61
14000
10000
elec.ts
8000
6000
4000
2000
Time
(a)
0 1000
diff(elec.ts)
−1500
Time
(b)
0.05 0.20
diff(log(elec.ts))
−0.15
Time
(c)
Figure 17: (a) Plot of Australian electricity production series; (b) Plot of the first order differ-
enced series; (c) plot of the first order differenced log-transformed series
62
Integrated model
63
ARIMA models
64
Examples
65
Simulation and fitting
Call:
arima(x = x, order = c(1, 1, 1))
Coefficients:
ar1 ma1
0.526 0.290
s.e. 0.038 0.042
66
IMA(1,1) model fitted to the Australian beer production
series
Call:
arima(x = beer.ts, order = c(0, 1, 1))
Coefficients:
ma1
-0.333
s.e. 0.056
> acf(resid(beer.ima))
67
Forecasts can be obtained using the predict function in R with the
parameter n.ahead set to the number of values in the future:
[1] 2365
68
0.8
0.4
ACF
0.0
−0.4
Lag
Figure 18: Australian beer series: Correlogram of the residuals of the fitted IMA(1,1) model
69
Seasonal ARIMA models
70
Examples
71
Fitting procedure
Once a best fitting model has been found the correlogram of the resid-
uals should be verified as white noise.
72
ARCH models: Example based on SP500 Series
> library(MASS)
> data(SP500)
> plot(SP500, type='l')
> acf(SP500)
73
2
SP500
−6 −2
Index
(a)
0.8
ACF
0.4
0.0
0 5 10 15 20 25 30 35
Lag
(b)
Figure 19: Standard and Poors SP500 Index: (a) Time plot; (b) Correlogram
74
Volatility
75
Correlogram of squared values
Since the mean should be approximately zero, the squared values repre-
sent the variance and the correlogram should pick up seriel correlation
in the variance process.
The correlogram of the squared values of the SP500 index is given by:
> acf(SP500^2)
From this we can see there is evidence of serial correlation in the squared
values, so there is evidence of volatility.
76
0.0 0.2 0.4 0.6 0.8 1.0
ACF
0 5 10 15 20 25 30 35
Lag
Figure 20: Standard and Poors SP500 Index: Correlogram of the squared values
77
Modelling volatility: ARCH model
where {wt} is white noise with zero mean and unit variance, and α0
and α1 are model parameters.
78
Extensions and GARCH models
where {wt} is again white noise with zero mean and unit variance.
79
GARCH model for SP500 series
Below the GARCH model is fitted to the SP500 return series. We check
the correlogram of the residuals and squared residuals, which should
be near white noise if we have obtained a good fitting GARCH model.
80
0.8
ACF
0.4
0.0
0 5 10 15 20 25 30 35
Lag
(a)
0.8
ACF
0.4
0.0
0 5 10 15 20 25 30 35
Lag
(b)
Figure 21: GARCH model fitted to SP500 series: (a) Correlogram of the residuals; (b) Correl-
ogram of the squared residuals
81
4. Multivariate models
Spurious regression
[1] 0.958
82
●
●
140000
●
●
●
●
●
Electricity production
100000
●
●
●
●
●
●
●
●
●●
●
60000
●
●
●
●
●
●
●
●
●
●
●
20000
●●
● ●
Chocolate production
Figure 22: Annual electricity and chocolate production plotted against each other
83
Spurious stochastic trends
The above can be repeated for different random number seeds though
you will only sometimes notice spurious correlation.
84
●
●
●
● ●
● ● ●
● ●
10
● ● ●
● ● ●● ●
● ●
● ● ●
●
● ●
●
●
● ●●
5
● ●
● ●● ● ●
●
●
●
y
● ● ●
●
●● ●
● ●
0
●●
●
●●
●
● ●
● ● ●
●● ● ●
●●
●
● ● ● ●
● ● ● ● ●
● ●●
● ● ●
−5
● ● ●
●
● ●● ● ● ●
●
●
●
● ●●
0 2 4 6 8 10 12
Figure 23: The values of two independent simulated random walks plotted against each other.
(See the code in the text.)
85
Stochastic trends are common in economic series, and so multiple eco-
nomic series require considerable care when trying to determine any
relationships between the variables.
86
Example: Exchange rates
The daily exchange rate series for UK pounds, the Euro, and New
Zealand dollars, given for the period January 2004 to December 2007,
are all per US dollar. The correlogram plots of the differenced UK and
EU series indicate that both exchange rates can be well approximated
by random walks, whilst the scatter plot of the rates show a strong
linear relationship which is supported by a high correlation of 0.95.
UK NZ EU
1 0.558 1.52 0.794
2 0.553 1.49 0.789
3 0.548 1.49 0.783
> acf(diff(xrates$UK))
> acf(diff(xrates$EU))
> plot(xrates$UK, xrates$EU, pch=4
> cor(xrates$UK, xrates$EU)
[1] 0.946
87
0.8
ACF
0.4
0.0
0 5 10 15 20 25 30
Lag
(a)
0.0 0.4 0.8
ACF
0 5 10 15 20 25 30
Lag
(b)
Figure 24: Correlograms of the differenced exchange rate series: (a) UK rate; (b) EU rate
88
0.56
UK rate
0.52
0.48
EU rate
Figure 25: Scatter plot of the UK and EU exchange rates (both rates are per US dollars)
89
Tests for unit roots
When analysing more than one financial series, the first step is usually
to see whether the series contain stochastic trends, and whether these
trends are common to more than one series.
> library(tseries)
> adf.test(x)
data: x
Dickey-Fuller = -2.23, Lag order = 4, p-value = 0.4796
alternative hypothesis: stationary
90
Phillips-Perron test
> pp.test(xrates$UK)
data: xrates$UK
Dickey-Fuller Z(alpha) = -10.6, Truncation lag parameter = 7, p-value =
0.521
alternative hypothesis: stationary
> pp.test(xrates$EU)
data: xrates$EU
Dickey-Fuller Z(alpha) = -6.81, Truncation lag parameter = 7, p-value =
0.7297
alternative hypothesis: stationary
91
Cointegration
The UK and Euro exchange rates are very highly correlated, which
is explained by the similarity of the two economies relative to the US
economy. So although the rates are approximately random walks, they
share stochastic trends due to the effect of the US economy.
Thus it is quite possible for two series to contain unit roots and be
related, so the relationship between them would not be spurious. Such
series are said to be cointegrated.
92
Cointegration: Definition
Both series are non-stationary but their difference {xt −yt} is stationary
since it is a finite linear combination of independent white noise terms.
Thus the linear combination of {xt} and {yt}, with a = 1 and b = −1,
produced a stationary series. Hence {xt} and {yt} are cointegrated,
and share the underlying stochastic trend {µt}.
93
Cointegration test in R
[1] 0.502
> adf.test(y)$p.value
[1] 0.544
data: cbind(x, y)
Phillips-Ouliaris demeaned = -1020, Truncation lag parameter = 9,
p-value = 0.01
The po.test provides evidence that the series are cointegrated since
the null hypothesis is rejected at the 1% level.
94
Exchange rate series
Below, there is evidence that the UK and Euro series are cointegrated,
so a regression model is justified in this case, and fitted using the lm
function.
The AR models provide a better fit to the residual series than the
ARIMA(1,1,0) model, so the residual series is stationary, supporting
the result of the Phillips-Ouliaris test, since a linear combination ob-
tained from the regression model has produced a stationary residual
series.
[1] -9880
[1] -9876
95
Bivariate white noise
Two series {wx,t} and {wy,t} are bivariate white noise when they are
white noise when considered individually but when considered as a
pair may be be cross-correlated at lag 0 (so cor({wx,t}, {wy,t}) may be
non-zero).
For more than two white noise series, the definition of bivariate white
noise readily extends to multivariate white noise.
96
Simulation of bivariate white noise
> library(mvtnorm)
> cov.mat <- matrix(c(1, 0.8, 0.8, 1), nr = 2)
> w <- rmvnorm(1000, sigma = cov.mat)
> cov(w)
[,1] [,2]
[1,] 1.073 0.862
[2,] 0.862 1.057
97
0.8
0.6
0.4
ACF
0.2
0.0
−20 −10 0 10 20
Lag
98
Vector Autoregressive Models
Two time series, {xt} and {yt}, follow a vector autoregressive process
of order 1 (denoted VAR(1)) if:
xt = θ11xt−1 + θ12yt−1 + wxt
yt = θ21xt−1 + θ22yt−1 + wyt
where {wxt} and {wyt} are bivariate white noise and θij are model
parameters. The name ‘vector autoregressive’ comes from the matrix
representation of the model:
Zt = ΘZt−1 + wt
where:
xt θ11 θ12 wxt
Zt = Θ= wt =
yt θ21 θ22 wyt
99
VAR estimation in R
x y
x 0.399 0.321
y 0.208 0.104
100
VAR model fitted to US economic series
Below a best fitting VAR model is fitted to the Gross National Product
(GNP) and Real Money (M1).
> library(tseries)
> data(USeconomic)
> US.ar <- ar(cbind(GNP, M1), method = "ols")
> US.ar$ar
, , GNP
GNP M1
1 1.11674 -0.0434
2 -0.00797 0.0633
3 -0.10774 -0.0188
, , M1
GNP M1
1 0.995 1.577
2 -0.194 -0.453
3 -0.802 -0.147
From the above, we see that the best fitting VAR model is of order 3.
The correlogram of the residual series indicates that the residuals are
approximately bivariate white noise, thus validating the assumptions
for a VAR model.
101
1.0
0.6
ACF
0.2
−0.2
0 5 10 15 20
Lag
(a)
1.0
0.6
ACF
0.2
−0.2
0 5 10 15 20
Lag
(b)
Figure 27: Residual correlograms for VAR(3) model fitted to US Economic Series: (a) Residuals
for GNP; (b) Residuals for M1
102
Prediction for VAR models
Below we give the predicted values for the next year of the series, which
are then added to a time series plot for each variable:
GNP M1
1988 Q1 3944 628
1988 Q2 3964 626
1988 Q3 3978 623
1989 Q4 3991 620
103
4000
3600
3200
Time
(a)
550
450
Time
(b)
Figure 28: US Economic Series: (a) Time plot for GNP (from 1981) with added predicted
values (dotted) for the next year; (b) Time plot for M1 (from 1981) with added predicted
values (dotted) for the next year
104
References
105