MAB210-2-13 Solutions Worksheet Section 4 1

MAB/MAN 210: Worksheet Solutions Week 4

A) Questions for Group Work (compulsory)

(These questions are in part A of the Worksheet for Section 4.)

1. (i) n
X
can take the values 0,1, 2, 3 with the probability transition matrix given
below
0.5 0.4 0.06 0.04
0.5 0.4 0.06 0.04
0 0.5 0.4 0.1
0 0 0.5 0.5
P
| |
|
|
=
|
|
\ .


(ii) The equations for solving
P t t ' ' =
are

0 1 2 3 0
0 1 2 3 1
0 1 2 3 2
0 1 2 3 3
0.5 0.5 0 0
0.4 0.4 0.5 0
0.06 0.06 0.4 0.5
0.04 0.04 0.1 0.5
t t t t t
t t t t t
t t t t t
t t t t t
+ + + =
+ + + =
+ + + =
+ + + =

as well as
1
3 2 1 0
= + + + t t t t


From the equations above we get 0 3 0 2 1 0
24 . 0 & 4 . 0 & t t t t t t = = =
. Hence
1 ) 24 . 0 4 . 0 1 1 (
0
= + + + t
and therefore
3788 . 0
0
= t
.

2. Transition probability matrix over states 0,1, 2, , 2 and 1 k k :

1 0 0 0
0 1 0 0
0 0 0 0
.

0 0 0 1
1 0 0 0
o o
o o
o
o o
o o
| |
|

|
|
|
|
|

|
|

\ .
P =

Stationary (or long-term) distribution
0 1 2 2 1
[ ]
k k
t t t t t

' = t satisfies
P t t ' ' = so that
0 1 0 1 0
0 1 1 1 0
1 2 2 2 1 0
(1 ) or ,
(1 ) or ,
(1 ) or ,
k k
t o t o t t t
t o t o t t t
t o t o t t t t

+ = =
+ = =
+ = = =


and so on. Therefore,
1
0 0 1 1
(since 1) for 0,1, , 1.
i k k
i k t t t t t

= = + + + = =
MAB210-2-13 Solutions Worksheet Section 4 2

B) Questions for further practice (voluntary)

(These questions can be found in part B) of the Worksheet for Section 4.)

1.

2. (i)


0 0.2 0.8
0.8 0.2 0
0.3 0.2 0.5
R O G
R
P O
G
| |
|
=
|
|
\ .


(ii) For steady-state:
Solve P t t ' ' =

0 0.8 0.3
0.2 0.2 0.2
0.8 0 0.5
R O G R
R O G O
R O G G
t t t t
t t t t
t t t t
+ + =
+ + =
+ + =


So,

1.6
0.52
0.8
G R
O R
t t
t t
=
=


And then
( ) 1.6 0.65 1 1 and 0.3077
R R
t t + + = = .

3.(i) Let 1, 2 denote paying the total amount, not paying the total amount. Then,

0.6 0.4
0.2 0.8
P
| |
=
|
\ .



and we need to solve P t t ' ' = such that

( ) ( )
0 1 0 1
0.6 0.4
0.2 0.8
t t t t
| |
=
|
\ .

MAB210-2-13 Solutions Worksheet Section 4 3


That is,

0 1 0
0 1 1
0.6 0.2
0.4 0.8
t t t
t t t
+ =
+ =


Now,
0 1 1 0
0.4 0.2 so that 2 t t t t = =

Therefore,
0 1
1/ 3, 2/ 3 t t = = . So 1/3 pay the total amount owing.

(ii) Let 0, 1, 2 denote paying the total, paying at least the minimum but not the total,
and not paying the minimum.

0.6 0.3 0.1
0.2 0.7 0.1
0.2 0.8 0
P
| |
|
=
|
|
\ .


We want the (2, 0) element of
2
P which is

( )
0.6
0.2 0.8 0 0.2 0.28
0.2
| |
|
=
|
|
\ .
.

That is, 28% of those who do not pay minimum, pay total in two months.

4. The elements of the matrix are conditional probabilities, with the first row being
conditional on the signal being 0 at the previous stage, and the second row
conditional on the signal being 1 at the previous stage. There are 10 zeroes as
possible starting points (last zero cannot be used in estimation), and 8 1s. Of
the 10 0s, only two are followed by a 1, so 2/10 estimates the probability of a
transition from 0 to 1. Working similarly to this we estimate the transition
matrix by

0 1

8 2
0
10 10
1 2 6
8 8
| |
|
|
|
|
\ .


MAB210-2-13 Solutions Worksheet Section 4 4
5. The number of items at the beginning of each day can take values 1,,K.

(i) Transition probability matrix over states 1, 2,,K is:

1 2 3 K-1 K

0 0 0 1
1 0 0 0
0 1 0 0
.
0 0 0 1 0
0 0 0 1
o o
o o
o o
o
o o
| |
|

|
|

|
|
|

|
|

\ .
1
2
3
P =
K - 1
K


(ii)
Stationary (or long-term) distribution
1 2 3 1
[ ]
K K
t t t t t

= t satisfies
' ' = P t t . That is,

2 1
2 3 2 2 3
3 4 3 3 4
1 1 1
1
(1 )
(1 ) i.e.
(1 )
(1 ) (1 )
K K K
K K
t o t
t o t o t t t
t o t o t t t
t o t o t o t
t o t o t

=
+ = =
+ = =
+ + =
+ =


From last equation,
1
(1 )
K
o
t t
o

=
Using this in the 2
nd
last equation gives
1 1 2 3 2
1
.....
K K
t t t t t
o

= = = = = .

Therefore we have

1 1 1 1
1
1 1 (1 )
( ... ) 1
( 1)
1
K
o
t t t t
o o o
t
o

+ + + + =

=

and
(1 )
1
K
K
o
t

=

.

Also,
2 1
1
.....
1
K
K
t t

= = =

.

MAB210-2-13 Solutions Worksheet Section 4 5
6. (i)
0 0 1
0.1 0.9 0
0.01 0.18 0.81
P
| |
|
=
|
|
\ .


For example,
( ) ( )
20 21
0.1 0.1, 0.1 0.9 0.9 0.1 ,... p p = = +

(ii) Solve
P t t ' ' =


0 1 2 0
0 1 2 1
0 1 2 2
0 0.1 0.01
0 0.9 0.18
1 0 0.81
t t t t
t t t t
t t t t
+ + =
+ + =
+ + =


So,
0 2 1 2
0.19 and 1.8 t t t t = = . Therefore,
( )
2 2
0.19 1.8 1 1 and 0.3344. t t + + = =

As such,
0
0.0635 t = which is the proportion of time the component is not working.