Ch12 2 Markov

Chapter 12
(12.6 12.8)
Markov Processes
1
12.6 Determining Limiting Behavior for

Markov Processes with Absorbing States
The steady state probability for each absorbing
state is the probability that the process is
eventually absorbed into that state.
To find steady states probabilities we need to
calculate the limiting transition matrix. This
matrix gives the eventual likelihood that the
process moves from each transient state to each
absorbing state.
2
DR. DALE BANDON Limiting Behavior

Step 1: Reconstruct the transition matrix. List
absorbing states first.

Paid up
Paid up
Absorbing
states
Non
absorbing
states
Sent to
collection
Between 45
and 90 days
overdue
Less than
45 days
overdue
Between 45
and 90 days
overdue
Sent to
collection
Less than
45 days
overdue
.45
.30
.55
.25
.05
.15
.25

Paid up
Paid up
Absorbing
states
Non
absorbing
states
Sent to
collection
0
Identity
Matrix ( I )
Between 45
and 90 days
overdue
A matrix of all
zeros
(0)
Less than
45 days
overdue
Between 45
and 90 days
overdue
Sent to
collection
Less than
45 days
overdue
.45 Transition 0
probabilities
between
Non-absorbing
.55 states and .05
Absorbing states (R)
.30 Transition .25

probabilities
between
Non absorbing
.15 states (Q) .25

:Step 2: Calculate the Fundamental Matrix N.
NN == (I(I -- Q)
Q)-1-1
1 0 .30 .25
N

0 1 .15 .25
.70 .25
.15 .75
15385
.
.5128
.
.3077 14359
The
The entries
entries in
in the
the fundamental
fundamental matrix
matrix represent
represent
the
the average
average number
number of
of times
times the
the process
process visits
visits
aa non-absorbing
non-absorbing state
state before
before visiting
visiting one
one of
of the
the
absorbing
absorbing states.
states.
6

Step 2: Calculate the Fundamental Matrix N.
State 3
State 3
State 4
15385
.
.3077
State 4
.5128
14359
.
An account currently less than 45 days over

due will visit this same state an average of
1.5385 times before being paid in full or turned
over to a collection agency.

Step 3: Calculate the Limiting Transition Matrix L.
LL == NN RR
The
The entries
entries in
in the
the Limiting
Limiting Transition
Transition matrix
matrix
represent
represent the
the probability
probability of
of reaching
reaching each
each
absorbing
absorbing state
state from
from each
each transient
transient state.
state.
15385
.
L NR
.3077
.5128 .45

14359
.
.55
0
.9744

.05
.9282
.0256
.0718
8
DR. DALE BANDON

The Limiting Transition Matrix
State 1
State 2
.9744 .0256
ttlyly
n
n
rrrree 4
State
u
u
.
9282
.
0718
tt cc e
n
n
u
e
u
bb
o
State 3
cccco wwililll
a
a
nn yss,,
a
a
t
t ddaay
a
a
h
tth 455
y
t
y
i
t
l
i
ibil haann 4 ulll.l.
b
aa s tth n ffu
b
b
o
rro leesss idd iin
p
p
ee uee l ppaai
h
h
TT errddu lllyly
oovve ennttuuaa
eevve
The rows in the L matrix

Correspond to nonabsorbing
states. The columns in the
L matrix correspond to
absorbing states.
DR. DALE BANDON

Limiting State Probabilities
Step 4: Calculate Limiting State Probabilities.
For a given state vector, the limiting state probability can be
calculated as follows:
Limiting
Limiting state
state vector
vector == (1)
(1) LL
.9744 .0256
(1)L = .6 .4
(.9559 .0441)
.9282 .0718
Suppose currently 60% of Dr. Bandons active account receivables are

less than 45 days overdue, and 40 % are between 45 and 90 days overdue.
On average, 95.59% of these accounts will ultimately be paid.
10
The Limiting Behavior Excel

=E4
Drag to Cell G9
=C2
Drag to Cell
D9
=E20
Drag to Cell F25
=C8-F8
Drag to Cell J9
=INDEX(MMULT($E$12:$F$
13,$C$4:$D$5),$G12,E$11)
Drag to Cell F21
11
The Limiting Behavior Excel

=INDEX(MINVERSE($I$8:$J$9),$G12,E$11)
drag toCell F13
=MMULT($B$24:$C$24,E24:E25)
Drag to Cell I24
12
DR. DALE BANDON

Mean time until absorption
Mean
Mean Time
Time Until
Until Absorption
Absorption ==
Sum
Sum of
of Values
Values in
in Rows
Rows of
of Fundamental
Fundamental Matrix
Matrix
An account currently less than 45 days overdue will be found in that
state on the average 1.5385 times, and in a state of between 45 and 90
days overdue 0.5128 times before being absorbed.
Less than 45
days overdue
Less than 45 days overdue
Between 45 and 90 days overdue
Between 45 and
90 days overdue
.5128
.3077
1.4359
1.5385
13
DR. DALE BANDON

Mean Time Until Absorption
Mean
Mean Time
Time Until
Until Absorption
Absorption ==
Sum
Sum of
of Values
Values in
in Rows
Rows of
of Fundamental
Fundamental Matrix
Matrix
Therefore, on the average, this account will be paid in full or sent for
collection on the average in 1.5385+.5128 = 2.05 months.
Less than 45
days overdue
Less than 45 days overdue
Between 45 and 90 days overdue
1.5385
.3077
Between 45 and
90 days overdue
.5128
= 2.05
1.4359
14
12.7 Using Markov Process in

Economic Analysis
Markov processes can be used to evaluate
aspects of a businesss economic performance.
The analysis is focused on the long run
characteristics of the business behavior.
15
ROLLEYS RENTALS -continued
The Profitability of a Business Venture

Rolleys Rentals daily profit is strongly related to
the weather in the city of Lahaina.
The daily profit has been estimated to be:
$120 on a sunny day
$40 on a cloudy day
$200 loss on a rainy day
Mr. Rolley would like to determine his expected

profit over the upcoming week if the weather in
Lahaina is sunny today.
16

Recall the transition matrix
Tomorrow's weather
Sunny
Sunny
Todays
weather
Cloudy
Rainy
Profit:
Cloudy
Rainy
0.75 0.20 0.05
0.45 0.40 0.15

0.35 0.45 0.20
$120
$ 40
$ - 200
If today is sunny the state vector for tomorrow is {0.75 0.20 0.05}
17

Tomorrow's weather
Sunny
Sunny
Todays
weather
Cloudy
Rainy
Profit:
Cloudy
Rainy
0.75 0.20 0.05
0.45 0.40 0.15

0.35 0.45 0.20
$120
$ 40
$ - 200
Expected profit for day 2 = .75(120) + .20(40) + .05(-200) = $88

18

The transition vector for day 2 is (2) = (1) P
0.75 0.20 0.05
{.75 .20 .05 } 0.45 0.40 0.15 = {.6700 .2525 .0775 }

0.35 0.45 0.20
Expected profit for day 3 =

.67(120) + .2525(40) + .0775(-200) = $75
19

The expected profit for the rest of the days was
calculated in a similar manner.
The resulting weekly expected profit is the sum

of the seven days daily expected profit.
Weekly expected profit = $508.30
See spreadsheet next
20
ROLLEYS RENTALS Excel Spreadsheet

=B8*$B$3+C8*$C$3+D8*$D$3
Drag to Cell F14
=SUM(F8:F14)
=$B7*F$2+$C7*F$3+$D7*F$4
Drag to cell D14
21
Determining the Value Of a Business

Mr. Rolley is considering selling his business
and needs an evaluation of its business value.
It is usual to evaluate a business such as
Rolleys at six times its expected yearly profit.
Mr. Rolley wishes to determine a fair asking
price to the business.
22
Determining the Value of a Business

From Excel we can get the following steady state probabilities for
weather in Lahaina:
Sunny: 0.6298, Cloudy: 0.2786, Rainy: 0.0916
Daily expected profit =
.6298 ($120) + .2786($40) + .0916(-$200) = $68.40
Annual expected profit = $68.40(365) = $24,966.
Business worth = 6($24,966) = $149,796.
Because businesses are usually sold at 5% to 10%

below their asking price, Mr. Rolley asks $160,000
for the business.
23
CRAFTMADE COMFORT BEDS - continued
Estimating Long-Term Expected Income

Sid Chase supervises 9 salespersons at Craftmade.
He earns $50 on each sale made by a salesperson.
Sid wants to estimate his expected income in the coming
month (21 working days).
24

Recall that states were defined over two consecutive days.
The states and their steady state probabilities are listed
Average number of Number of beds
below:
beds sold per day sold in two days

0
The salesperson sells no bed yesterday and 0none today
0.1228
The salesperson sells no bed yesterday and1/2
one bed today 1 0.2026
The salesperson
The salesperson
The salesperson
The salesperson
The salesperson
The salesperson
The salesperson
sells no bed yesterday and 1two beds today2

sells one bed yesterday and
1/2none today 1
sells one bed yesterday and1 one bed today2
3/2two beds today
3
sells one bed yesterday and
sells two beds yesterday and
1 no bed today2
3/2 one bed today
3
2 two beds today
4
0.0430
0.1496
0.1662
0.1274
0.0960
0.0744
25
0.0179

Recall that states were defined over two consecutive days.
The states and their steady state probabilities are listed
Average number of Number of beds
below:
beds sold per day sold in two days

of
0
TheNumber
salesperson
sells no bed yesterday and 0none today
0.1228
The
expected
number
of
sales
per
salesperson
sold The Probability
The expected
number
of sales
per
1/2
1 salesperson
Thebeds
salesperson
sells no bed yesterday
and
one
bed
today
0.2026
=
0(.1228)+(1/2)(.2026+.1496)+1(.0430+.1662
0
.1228
= 0(.1228)+(1/2)(.2026+.1496)+1(.0430+.1662
The salesperson sells no bed yesterday
and 1two beds today2 =0.0430
+.0960)+(3/2)(.1274+.0744)+2(.0179)
+.0960)+(3/2)(.1274+.0744)+2(.0179)
=.8198
.8198
1/2
.2026+.1496
The salesperson sells one bed yesterday and
0.1496
1/2none today 1
1
.0430+.1662+.0960
Sids
earning
for
from
The salesperson
sells one bed yesterday
and1 one
bed today
0.1662
2 days
Sidsexpected
expected
earning
for21
21
days
from
3/2
.1274+.0744
3/2two beds
3
the
==21(9)($50)(.8198)
The salesperson sells one bed yesterday
and
today
0.1274
the99salespersons
salespersons
21(9)($50)(.8198)
2
.0179
==$7747.11.
The salesperson
sells two beds yesterday
and
0.0960
1 no bed today2
$7747.11.
The salesperson sells two beds yesterday and

0.0744
3/2 one bed today
3
26
2 two beds today
4
The salesperson sells two beds yesterday and
0.0179
DR. DALE BANDON - continued
Estimating the Allowance for Doubtful Accounts

Dr. Bandon needs to estimate the allowance for
doubtful accounts before preparing his year-end
income tax return.
An appropriate allowance will consider the
average loss on all the accounts sent for
collection.
27

Data
Currently 554 accounts are outstanding.
342 accounts are less than 45 days overdue.
212 accounts are between 45 and 90 days overdue.
An account sent for collection has an average balance of
$286.
Dr. Bandon receives only 20% of the balance from each
account sent for collection.
28

The limiting probabilities between transient and
absorbing states were calculated in the L matrix before.
State 1 State 2
State 3
.9744 .0256
State 4
.9282 .0718
7.18% of the accounts

currently between 45 and 90
days overdue will ultimately
be sent for collection.
2.56% of the accounts

currently less than
45 days overdue will
ultimately be sent for
collection.
29

Expected number of accounts sent for collection =
.0256(342) +.0718(212) = 23.9768
Dr. Bandon loses 80% of the balance on an account sent for
collection = (80%)(286) = $228.80 on the average.
The allowance for doubtful accounts should be

(23.9768)($228.80) = $5485.89.
State 1 State 2
.9744 .0256
State 4 .9282 .0718

State 3
30
STACYS DEPARTMENT STORES - continued
Determining a Firms Actuarial Costs

Stacey's would like to know how much to budget
for this years pension plan.
Budgeting for the pension plan depends on the
number of years (transitions) an employee is
expected to stay with Staceys (in a transient
state) before leaving (an absorbing state).
31

Stacey's employs 1500 workers, 300 of whom
were promoted last year.
Current year contributions to the pension plan for
each employee = ($150) (Number of years the
employee is expected to remain with Stacey's).
32

...and the
to the sub-matrices I, 0, R, and Q
Recall
thepartitioning
transition matrix
Employee status next year
Retired
Retired
Employee
status
this
year
Quit
Fired
Promoted
Not Promoted
Quit
Not
Fired Promoted Promoted
1 0
0
0
0
0
0
1 0
0
.07 .12 .03 .32 .46
.
09
.
17
.
04
.
17
.
53
33

The fundamental matrix
1 0
.32 .46
N

.53
.17Promoted
0 1
.17
Not.47
promoted
.68
1
Promoted
.46
Not
promoted
19056
.
.7042
13256
.
1947
.
An employee who has been promoted last year is expected

to remain with Stacey's for 1.947+1.9056 = 3.8526 years
An employee who has not been promoted last year is expected
to be with Stacey's for .7024+1.3256 = 2.0298 years.
34

Stacey's should set aside for the pension plan:
(300)(3.8526)($150) =$173,367 for workers who
were promoted
(1200)(2.0298)($150)= $363,564 for workers who
were not promoted
Total funding for the pension plan = $563,931.
An employee who has been promoted last year is expected
to remain with Stacey's for 1.947+1.9056 = 3.8526 years
An employee who has not been promoted last year is expected
to be with Stacey's for .7024+1.3256 = 2.0298 years.
35
FAST-FOOD RESTAURANT SELECTION-continued
Market Share Analysis

Caesars Restaurants in Sandpoint considers three
possible strategies in an attempt to increase its market
share from the current 17.8%.
Offering dine-in customers an unlimited amount of free drink
refills.
Implementing a frequent diner program.
Increasing the amount of local couponing and print advertising.
Each program is estimated to affect the transition matrix,

incur some extra cost, and yield additional revenue.
36
Market Share Analysis

Data
40,000 customers are estimated to eat at the three fast-food
restaurants per week.
$.90 net profit per customer is estimated, excluding operating
costs
Weekly
Weekly Expenses
Expenses
Nonmanagerial
Nonmanagerial staff:
staff: (eight
(eight employees
employees @
@ $300
$300 per
per week)
week)
Assistant
Assistant manager
manager
Manager
Manager
Rent
Rent and
and utilities
utilities
Advertising/promotion
Advertising/promotion
Total
Total estimated
estimated weekly
weekly average
average operating
operating costs
costs
Total
Total ($)
($)
$2,400
$2,400
500
500
700
700
1,400
1,400
600
600
37
$5,600
$5,600
Market Share Analysis: Current Operation

The estimated weekly net profit =
(40,000)(.178) ($.90) - $5600 = $808
The current operation net profit is now compared

to the expected profit from implementing each
new strategy.
38
Market Share Analysis: Free Drink Refill

Changes caused by this strategy:
One fewer staff employee - reduction of $300 in fixed
costs per week to $5300.
Increased cost of $.14 per customer - net profit per
customer becomes $.76
A new transition matrix
.68
.34
.23
.21 .11
.46 .20
.29 .48
39
Market Share Analysis: Free Drink Refill

With Excel we find the following steady state
probabilities (long-run market share)
Rally Burger = .4788; Burger Barn = .3032; Caesars = .2179
With this strategy Caesars expected weekly profit

will be:
(40,000)(.2179)($.76) - $5300 = $1324.16
40
Market Share Analysis: Frequent diner program

Changes caused by this strategy
Additional cost of $200 for advertising per week.
Hire an additional staff person at at $300 per week
(weekly fixed cost becomes $5900).
A decline in average profit per customer to $.68.
Transition matrix will change to:
.65
.40
.20
.20
.40
.30
.15
.20
.50
41
Market Share Analysis: Frequent diner program

probabilities (long-run market share):

will be:
(40,000)(.2524)($.76) - $6100 = $765.28
42

Market Share Analysis: Coupons and Advertisement
Changes caused by this Strategy
An increase of $800 in the weekly advertising
expenses (weekly fixed costs become $6400).
Reduction in net profit per customer to $.78.
5% increase in total number of customers who eat in
fast food restaurants per week. .69 .18 .13
A new transition matrix
.36 .43 .21

.26 .32 .42
43

Market Share Analysis: Coupons and Advertisement
probabilities (long-run market share)
will be:
{(40,000)(1.05)}(.2281)($.78) - $6400 = $1072.40
44
Market Share Analysis: Summary

Free drink refills:
Net weekly profit = $1324.16
Frequent diner program:
Net weekly profit = $ 765.28
Increased couponing:
Net weekly profit = $1,072.40
The most profitable strategy is

to offer unlimited drink refills.
45
12.8 Applying Markov Processes to

Gambling Situations
Many business situations can be modeled as a
sequence of simple gambles.
Issues
Determining the effect of the initial stake on the
probability of reaching a goal.
Determining the effect the amount gambled has on
the probability of reaching a goal.
46
GAMBLING IN LAS VEGAS - continued

Tom Turner would like to study the effects that
different initial stakes have on his chances of
reaching $100, and on his playing time.
Tom starts with $50 on hand.
47

$0
Using Excel to solve$10
for
the limiting probabilities
$20
of the absorbing states
$30
for each initial stake we
$40
have:
$50
To ensure a better than
$60
50% chance of
reaching the goal of$70
$100 before going
$80
broke, Tom needs to
$90
start with at least $70
on hand.
$100
$0
$10
$20 $30
$40
.53
0
0
0
0
0
0
0
$50 $60
$70
$80
$90
$100
Initial
P(Tom's
stake
P(Tom
goes
Initial
P(Tom's
stake
P(Tom
goes
0
.47
0
0
0
0
0
0
0
0
Stake
bust)
Stake $$ reaches
reaches $100)
$100)
bust)
.53
0
.47
0
0
0
0
0
0
0
10
0.0595
0.9405
10
0.0595
0.9405
0 0.53 0
.47
0
0
0
0
0
0
20
0.1256
0.8744
20
0.1256
0.8744
0
0
.53
0
.47
0
0
0
0
0
30
0.1991
0.8009
30
0.1991
0.8009
0 40
0
0 0.2807
.53
0
.47 0.7193
0
0
0
0
40
0.2807
0.7193
0 50
0
0 0.3713
0
.53
0
.0.6287
47
0
0
0
50
0.3713
0.6287
0 60
0
0 0.4721
0
0
.53 0.5279
0
.47
0
0
60
0.4721
0.5279
0 70
0
0 0.5840
0
0
0
.0.4160
53
0
.47
0
70
0.5840
0.4160
0 80
0
0
0
0
0
0
.
53
0
.
47
0.7083
0.2917
80
0.7083
0.2917
0 90
0
0
0
0
0
0
0
0
1
0.8465
0.1535
48
90
0.8465
0.1535
To find the expected number of times Tom will

get to gamble, when making bets of $10 and
stopping when he winds up with either $100
or nothing, we observe the fundamental
matrix.
49

$10
$30
$40
$50
$60
$70
$80
$90
1.48 1.21 0.97 0.75 0.56 0.40 0.25 0.12
$20 1.67
. .
.
3.14 2.57 2.06 1.60 1.20 0.84 0.53 0.25
=7.7
$10 1.78
N=
$20
.
.
.
.
.
.
.
.
.
$80 0.58 1.09 1.54 1.94 2.30 2.62 2.90 3.14 1.48
$90
0.31 0.58 0.82 1.03 1.22 1.39 1.53 1.67 1.78

example:
Tom
starts
theplay
mean
number
of starts
times
The An
mean
numberIf of
times
Tomwith
will$10,
get to
roulette,
if he
have
$20 before
he stops
is 1.48.
with he
$10will
and
gambles
$10 each
play, playing
is the sum
of the first row entries
Tom will play the roulette 7.7 times on the average before stopping
if he starts with $10 and gambles $10 in each play.
50

Tom wants to know what initial stakes maximize the
number of times he plays roulette on the average.
Repeating the above procedure we find:
Initial
Initial stakes
stakes
Expctd
Expctd plays
plays
50
60
70
50
60
70
7.7
24.46 24.32
7.7 14.14
14.14 19.19
19.19 22.69
22.69 24.46
24.32 22.05
22.05
10
10
20
20
30
30
40
40
80
80
90
90
17.43
17.43 10.17
10.17
51

Tom would like to predict how changing the amount
he wagers each time affects his chances of reaching
$100, and the expected number of times he plays.
To maximize the
probability of reaching
his goal of $100 before
going broke...
But this policy will
minimize the number
of plays before
having to stop
To maximize the
number of times he
Gamble as Gamble as
plays roulette before
aggressively conservatively having to stop...
as possible
($50)
as possible
($1)
But this policy will

minimize the probability
of reaching the goal
before going broke
52
Copyright John Wiley & Sons, Inc. All rights

reserved. Reproduction or translation of this work beyond that
named in Section 117 of the United States Copyright Act
without the express written consent of the copyright owner is
unlawful. Requests for further information should be
addressed to the Permissions Department, John Wiley & Sons,
Inc. Adopters of the textbook are granted permission to make
back-up copies for their own use only, to make copies for
distribution to students of the course the textbook is used in,
and to modify this material to best suit their instructional needs.
Under no circumstances can copies be made for resale. The
Publisher assumes no responsibility for errors, omissions, or
damages, caused by the use of these programs or from the use
of the information contained herein.
53

Ch12 2 Markov

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ch12 2 Markov

Uploaded by

Copyright:

Available Formats

Chapter 12

12.6 Determining Limiting Behavior for

DR. DALE BANDON Limiting Behavior

DR. DALE BANDON Limiting Behavior

DR. DALE BANDON Limiting Behavior

.30 Transition .25

.15 states (Q) .25

DR. DALE BANDON Limiting Behavior

DR. DALE BANDON Limiting Behavior

An account currently less than 45 days over

DR. DALE BANDON Limiting Behavior

DR. DALE BANDON

The rows in the L matrix

DR. DALE BANDON

Suppose currently 60% of Dr. Bandons active account receivables are

The Limiting Behavior Excel

The Limiting Behavior Excel

DR. DALE BANDON

DR. DALE BANDON

12.7 Using Markov Process in

ROLLEYS RENTALS -continued

The Profitability of a Business Venture

Mr. Rolley would like to determine his expected

ROLLEYS RENTALS -continued

The Profitability of a Business Venture

0.75 0.20 0.05

0.45 0.40 0.15

ROLLEYS RENTALS -continued

The Profitability of a Business Venture

0.75 0.20 0.05

0.45 0.40 0.15

Expected profit for day 2 = .75(120) + .20(40) + .05(-200) = $88

ROLLEYS RENTALS -continued

The Profitability of a Business Venture

0.75 0.20 0.05

{.75 .20 .05 } 0.45 0.40 0.15 = {.6700 .2525 .0775 }

Expected profit for day 3 =

ROLLEYS RENTALS -continued

The Profitability of a Business Venture

The resulting weekly expected profit is the sum

ROLLEYS RENTALS Excel Spreadsheet

ROLLEYS RENTALS -continued

Determining the Value Of a Business

ROLLEYS RENTALS -continued

Determining the Value of a Business

Because businesses are usually sold at 5% to 10%

CRAFTMADE COMFORT BEDS - continued

Estimating Long-Term Expected Income

CRAFTMADE COMFORT BEDS - continued

Estimating Long-Term Expected Income

beds sold per day sold in two days

sells no bed yesterday and 1two beds today2

CRAFTMADE COMFORT BEDS - continued

Estimating Long-Term Expected Income

beds sold per day sold in two days

The salesperson sells two beds yesterday and

DR. DALE BANDON - continued

Estimating the Allowance for Doubtful Accounts

DR. DALE BANDON - continued

Estimating the Allowance for Doubtful Accounts

DR. DALE BANDON - continued

Estimating the Allowance for Doubtful Accounts

7.18% of the accounts

2.56% of the accounts