5.

Suppose that whether or not it rains today depends on previous

weather conditions through the last two days. That is, suppose what
if it has rained for the past two days, then it will rain tomorrow with
probability .7; if it rained today but not yesterday, then it will rain
tomorrow with probability .5; if it rained yesterday but not today,
then it will rain tomorrow with probability .4; if it has not rained the
past two days, then it will rain tomorrow with probability .2.
(a) Define all the states in detail so that this problem can be
formulated as a discrete time Markov chain.
(b) Set up the one-step transition probability matrix P.
(c) Given that it rained on Monday and Tuesday, what is the
probability that it will rain on Thursday?
<Sol>
(a) State 0: if it rained both today and yesterday,
State 1: if it rained today but not yesterday,
State 2: if it rained yesterday but not today,
State 3: if it did not rain either yesterday or today,
(b) 0 1 2 3

0 .7 0 .3 0
.5 0 .5 0
1  
P= 0
2 .4 0 .6
3 
0 .2 0

.8 

.7 0 .3 0 .7 0 .3 0
.5 0 .5 0 .5 0 .5 0
(c) P(2) = P2 = P‧P = 
0
  =
.4 0 .6 0 .4 0 .6
   
0 .2 0 .8  0 .2 0 .8 

.49 .12 .21 .18 

.35 .20 .15 .30 
 
.20 .12 .20 .48 
 
.10 .16 .10 .64 

M T W TH
ˇˇ
x ˇ P012
ˇ ˇ P002 P002 + P012 = .49+.12 = .61