MULTIVARIATE TIME SERIES ANALYSIS

AN ADAPTATION OF
BOX-JENKINS METHODOLOGY
Joseph N Ladalla
University of Illinois at Springfield, Springfield, IL

By inverting (1.2) using (1.1) it follows that the

underlying model for Xt is of the form,

KEYWORDS:
Multivariate time series, Box-Jenkins ARIMA
models, Principal components, Forecasting,
Eigenvalues, Eigenvectors.
SUMMARY:
A time domain analysis of Multivariate Time
series is suggested which exploits Box-Jenkins
methodology after the given time series data is
translated into the principal components based on
the dispersion matrix.
After fitting suitable
models for each of the principal components,
these models may be converted into
multivariate ARIMA models for the original
data. Similarly the forecasts for the individual
principal components may be put together into a
matrix of forecasts to obtain forecasts for the
original multivariate time series. We apply the
method on two examples - one simulated and the
other real data from Gregory C. Reinsel.
As a byproduct of this approach an
elegant method of determining the order of the
ARIMA model is provided for multivariate time
series.

(1.3) Xt = 1Xt-1 + at
where,
T
1 = P P , and = diag {1,1 , 2,1 ,.. , k,1 }
and at has MVN (0, ) with = PTDP;
2
2
2
D = diag( 1 , 2 ., k )
2.

MODEL ESTIMATION.

Now let X = { X 1,t, X2,t, , Xk,t}, t = 1, 2, , n

denote a multivariate time series data points
observed on the above stochastic process. Let
P denote the matrix of sample eigen-vectors of
the variance-covariance matrix of X and let
W = {w1,t, w2,t, , wk,t } denote the sample
principal components given by,
(2.1)

XP = W

We assume that wi,t is a realization from W i,t .

We now use Box-Jenkins methodology to fit a
model for each wi,t . We then proceed to obtain
the forecasts, say m-step ahead forecasts. Let
the matrix of forecasts for W be given by,

.
(2.2)

1. FROM MULTIVARIATE TO UNIVARIATE

ANALYSIS:

w11 w12 w13 w1k

Let {Xt } denote a multivariate stochastic process

where Xt is a k-variate normal vector. Let P
denote the matrix of eigen-vectors corresponding
to the variance covariance matrix of Xt. Next let
{Wt } denote the principal components (vectors) of
{Xt } given by,
(1.1)

Xt P = Wt

wj,t = I,1wj,t-1 + aj,t

wm1 wm2 wm3 wmk

Now from (2.1) and (2.2) we obtain the matrix of
forecasts for the original time series data given
by,
Xm
T

j = 1,2,, k

be the underlying model for W j , j=1,2,,k. Here

2
aj,t is assumed to have N(0, j ) distribution.
The components of Wt viz. W 1,t , , W k,t
independent univariate stochastic processes

are

w21 w22 w23 w2k

(2.3)

Let , for the sake of simplicity,

(1.2)

Wm

where P
matrix P.

= W m PT

denotes transpose of the

Similar operations are performed to obtain say,

95% confidence intervals for the individual
variates.

0.234943 0.863473
P = 0.904492 -0.362364
0.355943 0.350865

-0.446336
-0.224914
0.866140

3. MODEL FOR THE VECTOR XT

(4.1)

We illustrate hereunder how one may use

the foregoing to determine a model
for the
vector xt given sample data. For simplicity, let
us assume that k = 3 and let
wtT = ( ut vt wt)
and let, for simplicity, the fitted models for the
principal components ut, vt and wt be given by
ARIMA (1,0,1) of the form:

We fit appropriate ARIMA models for the

principal
components
using
Box-Jenkins
methodology.
Models for Ut, Vt, and W t
respectively are given by,
(4.2)
ut=40.22436+0.7985ut-1+at; at. N(0, 287.7)
vt=35.36+0. 37784vt-1 + bt; btN(0, 144.2)
wt = -7.54 +0.52677wt-1 + 0.23957 wt-12+ct
ct N( 0, 7.40)

ut = 1 + 1 ut-1 + at - 1 at-1
vt = 2 + 2 vt-1 + bt - 2 bt-1
wt = 3 + 3 wt-1 + ct - 3 ct-1

(3.1)

where at are i.i.d normal (0,12 ); bt are i.i.d

2
2
normal (0,2 ) and ct are i.i.d. normal (0,3 )
This yields a model for the vector wt given by,

Equivalently,

where wt = (ut vt wt)T ;

wt = + wt-1 + bt + bt-1
where,
T
= ( 1 2 3 )
T
bt = (at bt ct )

(3.2)

1
=

0
0

0 0
2 0
0 3

1
=

0
0

wt = + 1 wt-1 + 12 wt-12 + bt

(4.3)

= ( 40.23436 35.36 -7.54) ;

1 = diag ( 0.7985 0.37784 0.52677);

0 0

12 = diag( 0

2 0
0 3

T
bt = (at bt ct )

0.23957)

and bt is N(0, D)

(3.3) Observe that bt has normal MVN( 0, D)

where D = diag{12, 22, 32 }

and D = diag(287.66 144.16 7.40)

Now (2.1) and (3.2) yield a model for xt:

Now setting xt = ( xt yt zt )T, (2.1), (4.1), (4.2)

and (4.3) yield the model for xt given by,

(3.4)

xt = + xt-1 +at - at-1

(4.4)

where,
T
T
= P ; = P P ; = P P ; at = Pbt;

xt = + 1 xt-1 +12xt-12 + at
where,

at = Pbt ; at is N(0, );

= P = (43.35 25.274 20.197)

T
From (3.3) , (3.4) it follows that at is N( 0, PDP )

1 = P 1P

4. EXAMPLES:

Example 1.
T

12 = P 12P =

We simulate a trivariate (normally

distributed) time series data consisting of 105
observations. The data denoted by Xt , Yt , Zt are
given in Appendix.
Based on the dispersion
matrix we obtain the principal components of the
variables, using only the first 100 observations,
leaving the last 5 observations for comparison
with the forecasts. The principal components U t,
Vt and W t are independently and normally
distributed.
The matrix P of eigen-vectors is given by:

0.4307 0.1043 -0.0224

0.1043 0.7295 0.1064
-0.0224 0.1064 0.5429
0.0477 0.0240 -0.0926
0.0240 0.0121 -0.0467
-0.0926 -0.0467 0.1797
124.84 16.77
=
16.77

64.87
254.64

72.84
64.87

72.84

59.74
The matrices of 5-step ahead forecasts for U, V
and W, together with lower and upper 95%
confidence limits are given below:

a) Forecasts:
t
101
102
103
104
105

173.8353
179.0296
183.1772
186.4890
189.1335

64.9940
59.9175
57.9993
57.2746
57.0007

-30.8700
-31.9553
-32.4292
-33.2898
-33.4694

101
102
103
104
105

140.5930
136.4901
135.6501
136.0387
136.9046

41.4616
34.7612
32.6197
31.8633
31.5849

c)

W
-36.2033
-37.9833
-38.6362
-39.5456
-39.7386

95% upper confidence limits:

101
102
103
104
105

207.0776
221.5692
230.7044
236.9394
241.3623

88.5265
85.0737
83.3789
82.6859
82.4166

W
-25.5367
-25.9273
-26.2221
-27.0340
-27.2001

Now using (2.3) and (4.1)

we obtain the
corresponding forecasts versus the observed
values:

101
102
103
104
105

forecast

110.74
108.06
107.59
108.13
108.59

forecast

101
102
103
104
105

140.62
147.41
151.96
155.41
157.94

forecast

101
102
103
104
105

57.94
57.07
57.46
57.64
58.33

136.49
137.09
137.90
139.13
140.00

113.14
99.11
110.46
97.46
110.92

Y
conf. limits
lower
upper
120.28
119.40
119.56
120.39
121.32

160.96
175.40
184.35
190.43
194.52

33.23 82.65
27.88 86.26
26.26 88.66
25.35 89.93
25.39 91.25

63.15
49.63
53.58
44.87
52.02

Example 2.
Here we consider a real life example:
Monthly flour price indices for three U.S. cities .
(c.f Gregory C. Reinsel (l997), Table 9) We use
the first 95 of the 100 observations for analysis
holding the last 5 observations to compare with
the forecasts.
Since the procedure is a repetition of
the one adopted in the previous example, we
skip details to look only at the relevant facts:
Let

Observed

142.96
139.53
153.54
138.49
132.82

X = monthly flour price index for Buffalo,

Y = monthly flour price index for
Minneapolis.
Z = monthly flour price index for Kansas
city.

The matrix P of eigenvectors

and the
corresponding eigenvalues of the dispersion
matrix of X, Y, Z are given below:

X
conf. limits observed
lower
upper
84.99
79.04
77.28
77.13
77.17

Z
conf. limits observed
lower upper

The forecasts compare well with the exact..

b) 95% lower confidence limits :

t

P2

P1

P=

P3

0.529453 0.513652
0.573376 0.369889
0.625236 -0.774173

eigenvalues
Proportion of
variation

2425.77
0.9799

0.675160
-0.731041
0.098677

43.72
0.0177

6.01
0.0024

Using the matrices X and W for the matrices of

the original time series data and the matrix of
principal components, we have as in (2.1),
(4.5)

W = XP

Let as before, Ut , Vt, and W t denote the first,

second and third principal components. Under
the assumption that the original times series are
normally distributed, the principal components
are independently and normally distributed.
Using Box-Jenkins methodology, we fit ARIMA
models to the principal components. The fitted

models for Ut , Vt and W t are given below:

operating in the principal components space.

Models for Ut , Vt and W t are given by,

(4.6)
Ut = 1.2363Ut-1 - 0.2363Ut-2 + at
Vt = 0.7442Vt-1 - 0.2558Vt-2 +bt - 0.2064bt-3
+ 0.2204bt-7
W t = 0.5750W t-1 + ct

Finally using the forecasts for the principal

components, like in example 1, we obtain the 5step ahead forecasts together with 95%
confidence intervals for the original time series
given below:
X
95%
95%
t lower limit forecast upper limit

where ar is N (0,184.42), bt is N (0,8.56) and

ct is N (0,1.54)
The model may be put in compact matrix form as,
(4.7) wt = 1wt-1 - 2wt-2 + bt - 3 bt-3 + 7 bt-7
bt is N(0, D)

96
97
98
99
100

156.089
146.085
137.706
132.072
126.444

174.741
174.475
173.292
173.190
172.415

193.394
202.866
208.879
214.307
218.386

observed
187.5
190.7
190.4
192.4
192.9

where,
1 = diag (1.2363
2= diag ( 0.2363
3= diag (
0
7= diag (
0

0.7442 0.5750
0.2558
0
0.2064
0
0.2204
0
D= diag ( 184.42 8.56 1.54

)
)
)
)
)

And as in example 1, using (4.3) and (4.5)

we obtain a model for the vector Xt given by,

Y
95%
95%
t lower limit forecast upper limit
96
97
98
99
100

(4.8) xt = 1xt-1 + 2 xt-2 + at - 3 at-3 -7 at-7 ;

and at is N ( 0, ).
Here,
T
T
1 = P 1P ; 2 = P 2P ;
T
T
7 = P 7P ;
= PDP
Accordingly, we obtain;
(4.9)
0.8050
1 =
0.2329
0.1516

0.1516
0.1886
0.9349

0.1337 0.1203 -0.0235

0.1203 0.1127 0.0115
-0.0235 0.0115 0.2457

0.0545 0.0392 -0.0821

0.0392 0.028 2 -0.0591
-0.0821 -0.0591 0.1237

0.0581 0.0419 -0.0876

0.0419 0.0302 -0.0631
-0.0876 -0.0631 0.1321

166.782
166.097
165.155
164.823
164.082

182.358
191.410
197.203
202.180
206.005

Z
95%
95%
lower limit forecast upper limit

179.8
179.0
179.2
181.4
181.8

observed

3 = P 3P ;

0.2329
0.8156
0.1886

151.205
140.784
133.107
127.466
122.159

observed

54.657 56.852 57.748

56.852 62.624 63.551
57.748 63.551 77.239

Observe the ease with which the model (4.8) is

established using Box-Jenkins methodology

96
97
98
99
100

162.095
151.901
145.806
139.090
134.239

174.505
173.597
173.718
172.382
172.008

186.915
195.293
201.630
205.674
209.777

178.2
182.0
188.6
190.8
192.2

CONCLUSIONS:

By converting the original time series into

principal components it appears that one can
analyze the original series in the principal
component space by applying Box-Jenkins
methodology to the individual principal
components. If this procedure is acceptable
then we have an elegant method to analyze
multivariate time series both to establish a
model as well as to forecast.

One problem which needs mathematical

justification is that the principal components are
independent only up to 0-order
crosscorrelations. However, since identification and
estimation procedures depend upon the sample
acf and partial acf, which in turn depend on
higher order cross correlations it is hoped that
the problem does not arise.

APPENDIX:
Simulated trivariate time series data:
X
106.405
103.199
125.213
149.28
118.061
135.681
123.609
119.67
107.386
103.184
114.094
120.054
84.621
99.35
104.366
125.856
93.452
102.997
115.066
128.531
101.679
103.118
91.619
106.05
100.201
113.581
126.056
108.847
97.912
83.83
107.601
110.36
97.757
104.182
114.859
105.747
93.911
94.041
98.611
100.201
88.707
77.038

Y
190.627
186.567
216.202
192.67
182.183
178.08
183.006
169.606
179.878
155.704
144.206
166.466
155.456
131.834
121.94
109.621
117.757
124.693
156.691
132.815
129.522
138.485
145.72
142.91
147.96
152.793
154.536
165.975
136.182
143.16
145.963
153.876
163.029
148.128
157.864
152.697
149.361
145.982
138.33
107.725
135.912
147.826

Z
69.8236
64.0717
82.9996
86.1037
73.3294
79.8028
73.1775
71.5903
69.5109
59.1079
52.309
64.2461
46.125
45.9178
43.8563
49.5452
44.4521
48.3801
65.4364
66.0735
49.5604
49.2726
44.2546
51.4856
53.6439
58.5262
65.6212
58.81
47.0019
45.97
65.1901
70.4322
59.213
53.7676
58.5882
53.1915
45.1395
45.5159
48.2072
38.5165
47.7301
42.6818

88.54
101.416
115.772
101.246
114.453
113.326
125.198
126.252
120.437
126.726
110.813
112.05
108.752
106.33
118.326
124.521
124.204
108.119
97.963
93.941
89.856
85.456
99.458
101.673
117.352
117.677
114.288
121.491
113.204
100.379
115.707
111.279
105.084
107.303
115.289
125.418
104.866
99.596
116.503
106.761
100.369
115.777
107.297
110.316
96.653
84.422
114.216
109.858

Y
154.424
138.881
162.512
186.815
192.601
190.862
222.856
209.395
201.562
232.348
206.013
194.48
190.653
186.528
200.918
172.932
172.396
158.274
172.891
148.599
153.749
144.248
145.553
154.751
143.449
140.059
163.917
162.065
169.533
167.589
182.598
175.07
167.03
177.034
183.738
175.354
166.7
139.441
118.784
158.826
177.363
196.3
200.037
190.499
179.838
197.826
205.546
187.162

Z
50.2841
56.4294
71.8461
68.8396
79.7266
75.5322
86.5211
80.2245
70.4401
83.1272
71.6925
73.5274
65.5909
62.7813
71.8392
73.1649
74.4356
59.9918
59.4257
53.2992
50.943
46.2048
51.9204
55.9632
56.7054
52.1146
60.2398
61.0048
63.6008
54.1047
63.7947
58.8569
61.8931
66.3935
73.8751
72.9356
58.9892
44.6021
54.6768
53.6773
59.6366
71.5243
73.8049
65.213
61.2701
55.4614
70.3773
65.9143

X
124.751
122.008
125.239
104.81
119.74
126.335
140.156
116.937
120.667
119.445
113.135
99.113
110.462
97.458
110.919

Y
156.648
160.353
197.26
167.966
174.405
168.119
139.09
160.776
167.939
129.182
142.962
139.533
153.538
138.493
132.818

Z
66.1368
62.7405
75.3086
60.6377
71.4816
71.8745
76.9747
66.2836
69.1031
62.9973
63.149
49.6332
53.5748
44.8671
52.0201

REFERENCES

1.

Box, G.E.P and Jenkins, G.M. (1976).

Time Series Analysis. Prentice Hall,
New Jeresy.

2.

Brockwell, P.J. and Davis, R.A. (l991).

Time Series:Theory and Methods.
Springer-Verlag, New York.

3.

Johnson R.A., and Wichern D.W., (l992).

Applied Multivariate Statistical
Analysis. Prentice Hall, New
Jersey.