1 Solution of Psheet 11

1. (a) Here

 p, j = i + 1
pij = P (Xn+1 |Xn ) = q, j = i − 1
0 otherwise


(2n) 1
p00 = √ (4pq)n
nπ

So 0 is null recurrent if p = q = 1/2, otherwise transient (similar

for the other states).
(2n)
Since p00 → 0 as n → ∞, limiting distribution does not exist
and hence stationary distribution.
(b) The state space is S = {0, 1, . . . , N }.
 
1/N (N − 1)/N 0 ... 0
 0
 2/N (N − 2)/N . . . 0 

 .. .
.. .
.. .
.. ..
P = . .

 . 
 0 0 0 (N − 1)/N 1/N 
0 0 0 0 1
Here 0 is absorbing state and others are transient. So limiting
distribution and stationary distribution does not exist.

2. since 0 is the absorbing state, system can jump from the state j to
the state 0 by making self-loop in that state and coming back to the
previous state.
 
(n) n − 1 n−j j
∴ fj0 = p q n = j, j + 1, j + 2, . . .
j−1

3. S = C1 ∪ C2 ∪ {2}, where C1 = {0, 1}, C2 = {3, 4}

the stationary distribution of C1 is π0 = 1/4, π1 = 3/4, and for C2 is
π3 = 2/5 and π4 = 3/5.
(n)
For the last two parts, find µij = ∞
P
n=1 nfij

4. Consider the Markov chain with  state space S ={0, 1, 2, 3} with prob-
1/2 1/2 0
ability transition matrix P =  1/2 1/2 0  . Here 2 is transient
1/3 1/3 1/3
and 0, 1 are recurrent state, which have same limiting distribution as
stationary distribution.