MA225 - Tutorial 10 - Solutions

• 2.
P {Xn+1 = j|X0 , . . . , Xn−1 , Xn = i}
= P {f (Xn , Yn+1 ) = j|X0 , X1 , . . . , Xn = i}
= P {f (i, Yn+1 ) = j} = pij
The first equality follows since Xn+1 = f (i, Yn+1 ) given Xn = i.
The second equality is due to the fact that Yn+1 is independent of
(X0 , . . . , Xn ), because this vector is a function of (X0 , Y1 , . . . , Yn ),
which is independent of Yn+1 by assumption. The last equality fol-
lows by the definition of pij and fact that Yn+1 and Y1 have the same
distribution.

• 1. (a). Since Xn ’s are iid, {Xn } is a MC.

(b) Define (
1 if nth trail is head
Xn (ω) =
0 otherwise

Hence Yn = nk=1 Xk . Thus Yn+1 = Yn + Xn+1 = f (Yn , Xn+1 ). Now

P
by 2, {Yn } is a MC.
Similarly you need to answer the others questions. Try to use the fact
of question 2.

• 3. Assume X0 ≤ S and that X0 is independent of the Dn .

(
S − Dn Xn−1 ≤ s
Xn (x) =
Xn−1 − Dn s < i ≤ S

Thus {Xn } is a MC (according to 2).

Now from 2 we can easily derived
(
P (D1 = S − j) ,i ≤ s
pij =
P (D1 = i − j) ,s < i ≤ S

• 4. Let P (Yi = +1) = P =P1 − P (Yi = −1). {Yi } are iid. Now random
walk is defined as Xn = nk=1 Yk . Thus Xn+1 = Xn + Yn+1 . Using 2
we can show {Xn } is a MC.
( n
n+j−i n−j+i
(n) n+j−i p 2 q 2 when n + j − i = even
pij = 2
0 otherwise

Since there will n+j−i

2 be positive steps and n−j+i
2 negative steps in
order to go from i state to j state in steps when n + j − i is even.

1
• 5
Communicating classes are C(0) = {0, 2} = C(2), C(1) = {1}, C(3) =
{3}.
1 + f 2 + f 3 + · · · = 0.8 + (0.2)(0.1) + 0 = 1
F00 = f00 00 00
1 + f 2 + f 3 + · · · = 0.
F11 = f11 11 1
1 +f 2 +f 3 +· · · = 0+(1)(0.2)+(1)(0.8)(0.2)+(1)(0.8)2 (0.2)+
F22 = f11 11 1
··· = 1
1 + f 2 + f 3 + · · · = 0.3
F33 = f11 11 1
Thus the states 0 and 2 are recurrent and states 1 and 3 are transient.

• 6
 
1/2 1/2 0
P = 1/3 1/3 1/3
0 1/2 1/2

P (X0 = i) = 13 .
Hence P {X2 = 2, X0 = 1} = P (X2 = 2|X0 = 1)P (X0 = 1) =
p212 P (X0 = 1)
Now
 
5/12 5/12 1/6
P 2 = 5/18 8/18 5/12
1/6 5/12 5/12
Therefore p212 = 5/12 and P {X2 = 2, X0 } = 5/12 × 1/3 = 5/36.
P (X3 = 1|X1 = 0) = p201 = 5/12.
P (X2 = 2, X1 = 1|X0 = 2) = P (X0 = 2)P (X1 = 1|X0 = 2)P (X2 =
2|X1 = 1, X0 = 1) = 31 p21 p12 = 1/18.
P (X2 = 1) = P (X2 = 1|X0 = 0)P (X0 = 0) + P (X2 = 1|X0 =
1)P (X0 = 1) + P (X2 = 1|X0 = 2)P (X0 = 2) = 1/3(p201 + p211 + p221 ).
Similarly others can be done.
Now communicating classes, C(0) = C(1) = C(2) = {0, 1, 2}. Since
all are communicating to each other, the chain is not reducible or
irreducible.
(n)
Period: λ(i) = gcd{n : pii > 0}
λ(0) = λ(1) = λ(2) = 1
All the states are aperiodic.

2
Now stationary distribution π = [π1 π2 π3 ] is given by the solution of
πP = π.
Since the (finite) chain is irreducible and aperiodic, limiting distribu-
tion is same as stationary distribution.