STAT 153, FALL 2015

HOMEWORK 2 SOLUTIONS

Problem 1
(a)
√
√ var( nX̄)
P(| nX̄| ≥ ) ≤
2
√ 1 Pn
var( n[ n t=1 Xt ])
=
2
1 Pn
n var( t=1 Xt )
=
2
1 Pn Pn
n cov( t=1 Xt , t=1 Xt )
= 2

1
= (nRX (0) + (n − 1)RX (1) + (n − 2)RX (2) + . . . + RX (n − 1)
n2
+ (n − 1)RX (−1) + (n − 2)RX (−2) + RX (1 − n))
n  
1 X |h|
= 1− RX (h)
2 n
h=−n
∞
1 X
−→ RX (h) = 0
n→∞ 2
h=−∞
√
The inequality is Chebyshev’s. So we conclude that nX̄ →p 0.
It is also sufficient to show that the mean is zero for all n and the variance con-
verges to zero. The conclusion then follows from results in STAT134 or STAT135.
(b) There are many choices here. A general strategy for finding examples is to solve
RX (0) = 2 ∞
P
h=1 RX (h).
One example arises if we let Xt = Wt − Wt−1 where Wt ∼ W N (0, 1) then for
this process we have

∞
X
RX (h) = RX (0) + 2RX (1) = 2 − 2 = 0
h=−∞
It is also true that a constant process satisfies the equation, but the question
requires you to specify an equation with non-zero variance, so a constant process is
not a suitable example. This process is special because its characteristic polynomial
has a root on the unit circle.
1
2 STAT 153, FALL 2015 HOMEWORK 2 SOLUTIONS

Problem 2
(a) Suppose that |λ| > 1 then we can use induction to show Xt = ∞ k
P
k=0 λ Wt−k . On
the other hand if |λ| > 1 then the process is not causal, so we cannot write Xt in
terms of past Wt . However we also have:
1 1
Xt−1 = Xt − Wt
λ λ
So the reverse time process is causal and we can write
∞
X 1
Xt = Wt+k
(−λ)k
k=0
(b) Let Ut = (1 − λ2 B)Xt . Then
∞
X
Ut = (1 − λ1 B)−1 Wt = λl1 Wt−l .
l=0
On the other hand
∞
X 1
Xt = (1 − λ2 B)−1 Ut = − Ut+k
k=0
λk2
Combining these results yields
∞ ∞
X 1 X l
Xt = − k λ Wt+k−l
k=0
λ2 l=0 1
0 ∞
X λm
2
X λs
1
= Wm + Ws
λ
m=−∞ 2
− λ1 λ 2 − λ1
s=1

(c) Let z1 and z2 be the roots of the characteristic polynomial 1 − φ1 z − φ2 z 2 :

1 − φ1 z − φ2 z 2 = (1 − z1−1 z)(1 − z2−1 z)
= 1 − (z1−1 + z2−1 )z + z1−1 z2−1 z 2
So we have z1−1 + z2−1 = φ1 and −z1−1 z2−1 = φ2 .
(⇒) Suppose z1 and z2 are complex, then because the characteristic polynomial
has real coefficients they must occur in conjugate pairs: z1 = a + bi and z2 = a − bi
for a, b ∈ R.
φ1 + φ2 = z1−1 + z2−1 − z1−1 z2−1
1 1 1 1
= + −
a + bi a − bi a + bi a − bi
2a − 1
= 2
a + b2
a2
< 2 <1
a + b2
 STAT 153, FALL 2015 HOMEWORK 2 SOLUTIONS 3

φ2 − φ1 = −z1−1 z2−1 − z1−1 − z2−1

1 1 1 1
=− − −
a + bi a − bi a + bi a − bi
1 + 2a
=− 2
a + b2
a2
< 2 <1
a + b2

|φ2 | = | − z1−1 z2−1 | ≤ |z1−1 ||z2−1 | < 1

On the other hand if z1 and z2 are real then we deduce for z1 , z2 ∈ (−1, 1),

φ1 + φ2 = z1−1 + z2−1 − z1−1 z2−1

z1 + z2 − 1
=
z 1 z2
z1 z 2
<
z1 z2
=1

φ2 − φ1 = −z1−1 z2−1 − z1−1 − z2−1

z 1 + z2 + 1
=−
z1 z 2
z1 z2
< =1
z 1 z2

The argument for |φ2 | goes through without alteration from the complex case.
(⇐) We can deduce from the equations z1−1 + z2−1 = φ1 and −z1−1 z2−1 = φ2 that

    
1 1
q q
z1−1 , z2−1 2 2 2


= φ1 ± φ1 + 4φ2 , φ1 ∓ φ1 + 4φ 2
2 2

To prove the claim it suffices to show z1−1 and z2−1 fall inside the unit circle
because then the roots of the quadratic must fall outside the unit circle and we
have seen in lecture that this result implies causality.
4 STAT 153, FALL 2015 HOMEWORK 2 SOLUTIONS

When the discriminant is negative z1 and z2 are complex so we have

1
|z1−1 |2 = |z2−1 |2 = (φ21 + φ21 + 4φ22 )
4
1 2
= φ1 + φ22
2
1
< (1 − φ2 )2 + φ22
2
1 1
= + φ22
2 2
1 1
= + |φ2 |2 < 1
2 2
On the other hand if the discriminant is positive:
 
1 
q
−1

|z1 | =  φ1 ± φ21 + 4φ22 
2
1  p 
< φ1 ± (1 − φ2 )2 + 4φ2 

2
1  p 
= φ1 ± (1 + φ2 )2 

2
1
= |φ1 ± (1 + φ2 )| < 1
2
The last inequality follows from application of the first and second constraint.
 STAT 153, FALL 2015 HOMEWORK 2 SOLUTIONS 5

Problem 3
(a)
Xt − 0.5Xt−1 + 0.25Xt−2 = Wt + 0.5Wt−1
B B
⇒ (1 − √ )(1 − √ )Xt = (1 − 0.5B)Wt
−1 − 5 −1 + 5
Since there are no common factors we conclude that Xt is ARMA(2, 1). It is
immediate from the factorized
√ form that its characteristic polynomial for the AR
component has roots −1 ± 5. Since both of these roots avoid the unit circle we
also conclude the process is causal. The process is also invertible because the root
of the MA polynomial is 2, outside the unit circle.
(b) Rewriting Yt in terms of Xt in the second equation yields
Xt − Xt−1 + 0.5Xt−2 − 2(Xt−1 − Xt−2 + 0.5Xt−3 ) = Wt
⇒ Xt − 3Xt−1 + 2.5Xt−2 − Xt−3 = Wt
B B
⇒ (1 − 2B)(1 − )(1 − ) = Wt
1−i 1+i
So we conclude that this is an ARMA(3, 0) process. It is invertible but it is not
causal because 0.5 is a root of its AR characteristic polynomial.
(c)
Xt − 5/6Xt−1 + Xt−2 /6 = Wt − 6/8Wt−1 + Wt−2 /8
⇒ (1 − B/2)(1 − B/3)Xt = (1 − B/2)(1 − B/4)Wt
⇒ (1 − B/3)Xt = (1 − B/4)Wt
So we conclude that our process is a causal and invertible ARMA(1,1) process.
6 STAT 153, FALL 2015 HOMEWORK 2 SOLUTIONS

Problem 4
(a)

P (α1 Y1 + α2 Y2 |X) = µα1 Y1 +α2 Y2 + cov(X, X)−1 cov(X, α1 Y1 + α2 Y2 )

= α1 µY1 + α2 µY2 + cov(X, X)−1 [α1 cov(X, Y1 ) + α2 cov(X, Y2 )]
= α1 P (Y1 |X) + α2 P (Y2 |X)

(b)

t
Xt+1 = P (Xt+1 |X1 , . . . Xt )
= P (Xt + λXt−1 |X1 , . . . Xt )
= λP (Xt−1 |X1 , . . . Xt ) + P (Wt |X1 , . . . Xt )
= λXt−1 + 0
= ρ(1)Xt−1

The prediction does not improve as t increases as it is a function of Xt−1 alone.

This becomes clearer when we examine the prediction error:

t
E(|Xt+1 − Xt |2 ) = E(|Wt |2 ) = σ 2

(c)

(1 + θB)Wt = Xt
⇒ Wt = (1 + θB)−1 Xt
X
= (−θ)k Xt−k
k≥0

X X
Wt − (−θ)t+1 W−1 = (−θ)k Xt−k − (−θ)t+1 (−θ)k X−1−l
k≥0 l≥0
t−1
X
= (−θ)k Xt−k + (−θ)t X0
k=0
t−1
X
⇒ Wt = (−θ)k Xt−k + (−θ)t X0 + (−θ)t+1 W−1
k=0

(d) P (Wt+1 |X1 , . . . Xt ) = 0 because X1 , . . . Xt are all linear combinations of W0 , . . . Wt

alone and so they must be orthogonal to Wt+1 . A similar argument holds for W−1 .
 STAT 153, FALL 2015 HOMEWORK 2 SOLUTIONS 7

(e)
t
Xt+1 = P (Xt+1 |X1 , . . . Xt )
= P (Wt+1 + θWt |X1 , . . . Xt )
= θP (Wt |X1 , . . . Xt )
" t−1 #
X
= θP (−θ)k Xt−k + (−θ)t X0 + (−θ)t+1 W−1 |X1 , . . . Xt
k=0
" t−1 #
X
k t t
=θ (−θ) Xt−k + (−θ) X0
k=0
(f)
(g)