You are on page 1of 3

# IEOR 6711: Stochastic Models I

## Fall 2012, Professor Whitt

Solutions to Homework Assignment 12 due on Thursday, November 29
Problem 5.3 (a) Let N(t) denote the number of transitions be t. It is easy to show in this case
that
P(N(t) n)

j=n
e
Mt
(Mt)
j
j!
and thus P(N(t) < ) = 1.
(b) Let X
n+1
denote the time between the n-th and (n+1)-st transition and let J
n
denote
the n-th state visited. Also if we let
N(t) sup{n : X
1
+ + X
n
t}
then N(t) denotes the number of transitions by t. Now let j be the rst recurrent state
that is reached and suppose it was reached at the n
0
-th transition (n
0
must be nite
by assumption). Let n
1
, n
2
, be the successive integers n at which J
n
= j. (Such
integers exist since j is recurrent.) Set T
0
= X
1
+ + X
n
0
, and
T
k
X
n
k1
+1
+ + X
n
k
.
In other words, T
k
denote the amount of time between the k-th and (k +1)-th visit to
j. Therefore it follows that {T
k
, k 1} forms a renewal process, and so

k=1
T
k
=
with probability 1. Since

n=1
X
n
=

k=0
T
k
it follows that

n=1
X
n
= .
Problem 5.4 Let T
i
denote the time to go from i to i + 1, i 0. Then

N1
i=0
T
i
is the time to
go from 0 to N. Now T
i
is exponential with rate
i
and the T
i
are independent. Hence
E
_
e
s

N1
i=0
T
i
_
=
N1

i=0

i
s
.
We may use it to compute the mean and variance or we can do directly and mean =
1/
0
+ + 1/
N1
, variance = 1/
2
0
+ + 1/
2
N1
.
1
Problem 5.9
P
ij
(t + s) =

k
P(X(t + s) = j | X
0
= i, X(t) = k)P(X(t) = k | X
0
= i)
=

k
P
kj
(s)P
ik
(t) .
Problem 5.10 (a)
lim
t0
1 P(t)
t
= v
0
.
(b) The rst inequality follows from exercise 5.9 and
P(t + s) = P(X(t + s) = 0|X(0) = 0, X(s) = 0)P(s)
+P(X(t + s) = 0|X(0) = 0, X(s) = 0)(1 P(s))
P(t)P(s) + 1 P(s) .
(c) From (b)
P(s)P(t s) P(t) P(s)P(t s) + 1 P(t s)
or
P(s) + P(t s) 1 P(t) P(s) + 1 P(t s)
where the left hand inequality follows from
P(s)(1 P(t s)) 1 P(t s) .
lim
st
P(s t) = 1 implies the continuity of P.
Problem 5.13
i+k1

j=i

j
+
j
.
Problem 5.15 (a) Birth and death process.
(b)
n
= n + ,
n
= n.
(c) Set M(t) = E[X(t)|X(0) = i]. Then
E[X(t + h)|X(t)] = X(t) + (X(t) + )h X(t)h + o(h)
and so M(t+h) = M(t)+()M(t)h+h+o(h). Therefore, M

(t) = ()M(t)+
or e
()t
[M

(t) ( )M(t)] = e
()t
. Integrating both sides gives
e
()t
M(t) =

e
()t
+ C
2
or
M(t) = Ce
()t

.
As M(0) = i we obtain
M(t) =

_
e
()t
1
_
+ ie
()t
.
Problem 5.21 With the number of customers in the shop as the state, we get a birth and death
process with
0
=
1
= 3,
1
=
2
= 4. Therefore P
1
=
3
4
P
0
, P
2
=
3
4
P
1
=
_
3
4
_
2
P
0
. And
since P
0
+ P
1
+ P
2
= 1, we get P
0
= 16/37.
(a)
P
1
+ 2P
2
=
_
3
4
+ 2
_
3
4
_
2
_
P
0
=
30
37
(b) The proportion of customers that enter the shop is
(1 P
2
)

= 1 P
2
= 1
9
37
=
28
37
.
(c) = 8, and so P
0
=
64
97
. So the proportion of customers who now enter the shop is
1 P
2
= 1
_
3
8
_
2
64
97
=
88
97
.
The rate of added customers is therefore

88
97

28
37
0.45 .
The business he does would improve by 0.45 customers per hour.
3