Instructions

Sect.
Link to Worksheet

Computer Matrix
Computer Economics
Computer Matrix
Computer Economics
Computer Matrix
Computer Economics
Computer Matrix
Computer Economics
Brand Matrix
Brand Economics
Brand Transient
Brand Steady-State
IRS Matrix
IRS Economics
IRS Transient

2
Broker Matrix
Broker Economics
Broker Transient
Broker Steady_State
Broker Probabilities

Page 1

Instructions

Road Matrix
Road Economics
Road Transient
Road Steady_State

Ex P2 Matrix
Ex P2 Economics
Ex P2 Steady-State
Ex P2 Probabilities
Ex P2 Transient

Page 2

Instructions

Chapter 12
Operations Research Models and Methods
Paul A. Jensen and Jonathan F. Bard

Discrete Time Markov Chain Models-1

http://www.me.utexas.edu/~jensen/ORMM/computation/unit/stoch_anal/index.html
http://www.me.utexas.edu/~jensen/ORMM/computation/unit/rvadd/index.html

Instructions
To use this workbook, you must first load the Markov Analysis add-in. See the Excel help file
topic "About add-ins" before continuing.
Some problems use the the Random Variables add-in. Also load that add-in.
This workbook comes without control buttons. To make the buttons choose the Add Buttons
commands from the Markov Analysis menu.
The Random Variables add-in uses functions defined by the add-in. Use the Relink Functions
command to establish the proper links.

Discrete Time Markov Chain Models

Transition Matrix
Computer Repair Example
Matrix
Economics

Computer Repair Example - Repairman works on one machine at a time

Matrix
Economics

Computer Repair Example - Repair operation takes two days

Matrix
Economics

Computer Repair Example - Repair operation takes two days but computers fail
Matrix
Economics

Brand Switching
Matrix
Economics
Transient
Steady-State Analysis

IRS auditing example

Matrix
Economics
Transient

Multi-Step Transitions
Stock Broker
Matrix
Economics
Transient
Steady-State Analysis
Probabilities

Economic Analysis

Page 3

Instructions

Road Maintenance
Matrix
Economics with discounted cash flow analysis
Transient
Steady-State Analysis

Example Problem that is not included in the text chapter. It has two transient st
Matrix
Economics
Steady-State Analysis
Probabilities
Transient
Section 5 Applications are in the c12_dtmc_models2.xls workbook.

Page 4

Instructions

Add-ins Used
Markov Analysis - Markov Chain
Markov Analysis - Markov Chain
Markov Analysis - Markov Chain
Markov Analysis - Markov Chain
Markov Analysis - Markov Chain
Markov Analysis - Markov Chain

t computers fail independently

Markov Analysis - Markov Chain
Markov Analysis - Markov Chain
Markov
Markov
Markov
Markov

Analysis
Analysis
Analysis
Analysis

Markov
Markov
Markov
Markov

Chain
Chain
Chain
Chain

Markov Analysis - Markov Chain

Markov Analysis - Markov Chain
Markov Analysis - Markov Chain

Markov
Markov
Markov
Markov
Markov

Analysis
Analysis
Analysis
Analysis
Analysis

Markov
Markov
Markov
Markov
Markov

Chain
Chain
Chain
Chain
Chain

Page 5

Instructions

Markov
Markov
Markov
Markov

Analysis
Analysis
Analysis
Analysis

Markov
Markov
Markov
Markov

Chain
Chain
Chain
Chain

Markov
Markov
Markov
Markov
Markov

Chain
Chain
Chain
Chain
Chain

two transient states.

Markov
Markov
Markov
Markov
Markov

Analysis
Analysis
Analysis
Analysis
Analysis

Page 6

Markov Chain Transition Matrix

Type: DTMC
Title: Comp1

Step
CalculateMeasure
Change
Day
Analyze
State
Index Names
Economics
0
0_fail
1
1_fail
Transient
2
2_fail
Steady State
n-step Probabilities
First Pass
Simulate
Absorbing States

3
0_fail
1_fail
2_fail
Sum

Matrix Analyzed.
3 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
0_fail
1_fail
2_fail
0.6
0.3
0.1
0.8
0.2
0
1
0
0
2.4
0.5
0.1

Sum

Status
1 Class-1
1 Class-1
1 Class-1

Economic Data

Discount
Type: DTMC EconomicRate per
Title: Comp1 Measure
Day
Cost
0.1
0
0_fail
Calculate
1
1_fail
2
2_fail
Matrix

Transition Cost Matrix

State Combined
0
1
2
Cost
Cost
0_fail
1_fail
2_fail
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

on Cost Matrix

Markov Chain Transition Matrix

Type: DTMC
Title: Comp2

Step
CalculateMeasure
Change
Day
Analyze
State
Index Names
Economics
0
0_fail
1
1_fail
Transient
2
2_fail
Steady State
n-step Probabilities
First Pass
Simulate
Absorbing States

3
0_fail
1_fail
2_fail
Sum

Matrix Analyzed.
3 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
0_fail
1_fail
2_fail
0.6
0.3
0.1
0.8
0.2
0
0
1
0
1.4
1.5
0.1

Sum

Status
1 Class-1
1 Class-1
1 Class-1

Economic Data

Discount
Type: DTMC EconomicRate per
Title: Comp2 Measure
Day
Cost
0.1
0
0_fail
Calculate
1
1_fail
2
2_fail
Matrix

Transition Cost Matrix

State Combined
0
1
2
Cost
Cost
0_fail
1_fail
2_fail
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

on Cost Matrix

Markov Chain Transition Matrix

Type: DTMC
Title: Comp3

Step
CalculateMeasure
Change
Day
Analyze
State
Index Names
Economics
0
S00
1
S10
Transient
2
S20
3
S11
Steady State
4
S21
n-step Probabilities
First Pass
Simulate
Absorbing States

5
S00
S10
S20
S11
S21
Sum

Matrix Analyzed.
5 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
S00
S10
S20
0.6
0.3
0
0 -1E-008
0.8
0.8
0.2 -1E-008
0
0
0
0
1
0
1.4
1.5
0.8

3
S11
0.1
0
0
0
0
0.1

4
S21
0
0.2
0
1
0
1.2

Sum
1
1
1
1
1

Status
Class-1
Class-1
Class-1
Class-1
Class-1

Economic Data

Discount
Type: DTMC EconomicRate per
Title: Comp3 Measure
Day
Cost
0.1
0
S00
Calculate
1
S10
2
S20
Matrix
3
S11
4
S21

State Combined
Cost
Cost
0
0
0
0
0
0
0
0
0
0

0
S00
0
0
0
0
0

Transition Cost Matrix

1
2
S10
S20
0
0
0
0
0
0
0
0
0
0

on Cost Matrix
3
S11

4
S21
0
0
0
0
0

0
0
0
0
0

Markov Chain Model

Define States

Name: Comp4
Type: Network
State Var.:
1
Event Var.:
1

Economic Measure = Cost

Fill Matrix
Add Transitions

State Vector
Event Vector
Define the state vector and logical conditions for feasi Define the event vector and logical conditions for feas
Step Feas.
Step Feas.
1
1
State
4
Event
9
Feas.
1
1
1
Feas.
1
1
1
Min
0
Min
1
Max
4Formula for state names
Max
9Formula for event names
4
5
S21
8
9
0/0 fail
5
1
9
1
Formula for state Cost
Definitions
0
Definitions
s1: Node Index
e1: Arc Index

State-Event Transition Equations, Probabilities and Costs

Description:
State
1
Event
1
Min
0
Min
1
Max
4
Max
9
1
1
1
1
Logic
s1
Index Name
Trans. Prob.
Trans. Cost
Next
1
1
2 S10
1
0

Index
1
2
3
4
5

s1
0
1
2
3
4

NameMeasureMeasure
1
Measure
2
3
S00
0
0
0
S10
0
0
0
S20
0
0
0
S11
0
0
0
S21
0
0
0

d logical conditions for feasible events.

ula for event names

0/0 fail

Repair operation takes two days but computers fail ind

Network Model
The RV add-in must be installe
Build Matrix Model Random Variable
Distribution
trials (n)
probability (p)
Number of States
Number of Events
Economic Measure
Time Measure

=
5
=
9
= Cost
= Day

State Transition Network Data

State Data
Event Data
State Name Cost
Event
Name
0
S00
0
1
0/2 fail
1
S10
0
2
1/2 fail
2
S20
0
3
2/2 fail
3
S11
0
4
0/1 fail
4
S21
0
5
1/1 fail
6
0/1 fail
7
1/1 fail
8
0/0 fail
9
0/0 fail

ys but computers fail independently

RV add-in must be installed for this model
RV1
Binomial
2
0.2

From
0
0
0
1
1
2
2
3
4

To
0
1
3
2
4
0
1
4
1

Prob.
0.64
0.32
0.04
0.8
0.2
0.8
0.2
1
1

Cost
0
0
0
0
0
0
0
0
0

Markov Chain Transition Matrix

Type: DTMC
Title: Comp4

Step
CalculateMeasure
Change
Day
Analyze
State
Index Names
Economics
0
S00
1
S10
Transient
2
S20
3
S11
Steady State
4
S21
n-step Probabilities
First Pass
Simulate
Absorbing States

5
S00
S10
S20
S11
S21
Sum

Matrix Analyzed.
5 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
S00
S10
S20
0.64
0.32
0
0 -1E-008
0.8
0.8
0.2 -1E-008
0
0
0
0
1
0
1.44
1.52
0.8

3
S11
0.04
0
0
0
0
0.04

4
S21
0
0.2
0
1
0
1.2

Sum
1
1
1
1
1

Status
Class-1
Class-1
Class-1
Class-1
Class-1

Economic Data

Discount
Type: DTMC EconomicRate per
Title: Comp4 Measure
Day
Cost
0.1
0
S00
Calculate
1
S10
2
S20
Matrix
3
S11
4
S21

State Combined
Cost
Cost
0
0
0
0
0
0
0
0
0
0

0
S00
0
0
0
0
0

Transition Cost Matrix

1
2
S10
S20
0
0
0
0
0
0
0
0
0
0

on Cost Matrix
3
S11

4
S21
0
0
0
0
0

0
0
0
0
0

Markov Chain Transition Matrix

Type:
Title:

DTMC
Brand

Step
CalculateMeasure
Change
Week
Analyze
State
Index Names
Economics
0
B1
1
B2
Transient
2
B3
Steady State
n-step Probabilities
First Pass
Simulate
Absorbing States

3
B1
B2
B3
Sum

Matrix Analyzed.
3 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
B1
B2
B3
0.9
0.07
0.03
0.02
0.82
0.16
0.2
0.12
0.68
1.12
1.01
0.87

Sum

Status
1 Class-1
1 Class-1
1 Class-1

Economic Data
Type:
Title:

Discount
DTMC EconomicRate per
Brand Measure Week
Cost
0.1
0
B1
Calculate
1
B2
2
B3
Matrix

State Combined
Cost
Cost
0
0
0
0
0
0

0
B1

Transition Cost Matrix

1
2
B2
B3
0
0
0
0
0
0
0
0
0

on Cost Matrix

Transient Analysis
Type:
Title:

DTMC
Brand

Start
More
Chart
Matrix

Back

Initial
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

Average Cost
0
1
B1
B2
0
1
0.02
0.82
0.0664
0.693
0.12174 0.60178
0.1769 0.53516
0.2275 0.48577
0.27181 0.44866
0.30951 0.42047
0.34097 0.39886
0.36689 0.38215
0.38803 0.36916
0.40517 0.35901
0.419 0.35105
0.43011 0.34478
0.43902 0.33984
0.44614 0.33594
0.45183 0.33285
0.45637 0.3304
0.45998 0.32846
0.46287 0.32693
0.46516 0.3257

0 Discounted Cost
2
Step
Cum.
B3
Cost
Cost
0
0
0
0.16
0
0
0.2406
0
0
0.27648
0
0
0.28794
0
0
0.28673
0
0
0.27953
0
0
0.27002
0
0
0.26017
0
0
0.25096
0
0
0.24281
0
0
0.23581
0
0
0.22995
0
0
0.2251
0
0
0.22114
0
0
0.21792
0
0
0.21532
0
0
0.21323
0
0
0.21155
0
0
0.21021
0
0
0.20914
0
0

0
NPW
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

Steady State Analysis

Type: DTMC
Title: Brand
Solve by: Inverse

Matrix

Average
0
1
2
Cost
B1
B2
B3
per period
Steady State Dist. 0.47407 0.32099 0.20494
0

Markov Chain Transition Matrix

Type:
Title:

DTMC
IRS

Step
Matrix changed. Press Analyze button before lea
CalculateMeasure
Change
Year
Analyze
State
2
0
1
Index Names
Audit
No-Audit
Sum
Economics
0
Audit
Audit
0.6
0.4
1
1
No-Audit No-Audit
0.5
0.5
1
Transient
Sum
1.1
0.9
Steady State
n-step Probabilities
First Pass
Simulate
Absorbing States

ess Analyze button before leaving.

Status
Class-1
Class-1

Economic Data
Type:
Title:

Discount
DTMC EconomicRate per
IRS
Measure Year
Cost
0.1
0
Audit
Calculate
1
No-Audit
Matrix

Transition Cost Matrix

State Combined
0
1
Cost
Cost
Audit
No-Audit
0
0
0
0
0
0
0
0

on Cost Matrix

n-Step Transition Matrix

Type:
Title:

DTMC
IRS
Start
More
Matrix

Steps/Iter
0
1

1
Audit
No-Audit

1 step Transition Matrix 1 step Cost

0
1
Average
Audit
No-Audit
per step
0.556
0.444
0
0.555
0.445
0

Present
Worth
0
0

Markov Chain Transition Matrix

Type:
Title:

DTMC
Broker

Step
CalculateMeasure
Change
Week
Analyze
State
Index Names
Economics
0
a
1
b
Transient
2
c
Steady State
n-step Probabilities
First Pass
Simulate
Absorbing States

3
a
b
c
Sum

Matrix Analyzed.
3 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
a
b
c
0
0.5
0.5
0.75
0
0.25
0.75
0.25
0
1.5
0.75
0.75

Sum

Status
1 Class-1
1 Class-1
1 Class-1

Economic Data
Type:
Title:

Discount
DTMC EconomicRate per
Broker Measure Week
Cost
0.1
0
a
Calculate
1
b
2
c
Matrix

State Combined
Cost
Cost
1700
1450
1450 1187.5
1600 1387.5

Transition Cost Matrix

0
1
2
a
b
c
0
-200
-300
-250
0
-300
-200
-250
0

on Cost Matrix

Transient Analysis
Type:
Title:

DTMC
Broker

Start
More
Chart
Matrix

Back

Initial
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

Average Cost
0
1
a
b
0.3
0.5
0.525
0.2
0.35625 0.33125
0.48281 0.25625
0.38789 0.30664
0.45908 0.27031
0.40569 0.29719
0.44573 0.27712
0.4157 0.29215
0.43823 0.28089
0.42133 0.28933
0.434
0.283
0.4245 0.28775
0.43163 0.28419
0.42628 0.28686
0.43029 0.28486
0.42728 0.28636
0.42954 0.28523
0.42785 0.28608
0.42912 0.28544
0.42816 0.28592

1423.2
2
c
0.2
0.275
0.3125
0.26094
0.30547
0.27061
0.29712
0.27714
0.29215
0.28089
0.28933
0.283
0.28775
0.28419
0.28686
0.28486
0.28636
0.28523
0.28608
0.28544
0.28592

Discounted Cost
Step
Cum.
Cost
Cost
1306.25 1306.25
1380.31 2686.56
1343.52 4030.08
1366.43 5396.5
1350.42 6746.92
1362.13 8109.05
1353.42 9462.47
1359.93 10822.4
1355.05 12177.5
1358.71 13536.2
1355.97 14892.1
1358.03 16250.2
1356.48 17606.6
1357.64 18964.3
1356.77
20321
1357.42 21678.5
1356.93 23035.4
1357.3 24392.7
1357.03 25749.7
1357.23
27107
1357.08
28464

11704
NPW
1187.5
2328.25
3337.66
4270.94
5109.45
5878.33
6572.85
7207.27
7781.94
8305.78
8781.04
9213.75
9606.68
9964.18
10289
10584.4
10852.9
11097
11318.9
11520.6
11704

Steady State Analysis

Type: DTMC
Title: Broker
Solve by: Inverse

Matrix

Average
0
1
2
Cost
a
b
c
per period
Steady State Dist. 0.42857 0.28571 0.28571 1357.1429

n-Step Transition Matrix

Type:
Title:

DTMC
Broker
Start
More
Matrix

Steps/Iter
0
1
2

1
a
b
c

2 step Transition Matrix

0
1
2
a
b
c
0.75
0.125
0.125
0.1875 0.4375
0.375
0.1875
0.375 0.4375

2 step Cost
Average
per step
1368.75
1310.94
1385.94

2 step Cost
Present
Worth
2382.23
2264.98
2405.48

Markov Chain Transition Matrix

Type:
Title:

DTMC
Road

Step
Matrix Analyzed.
CalculateMeasure
4 Recurrent States
Change
Year
1 Recurrent State Class
Analyze
0 Transient States
State
4
0
1
2
Index Names
New
Degraded Poor
Economics
0
New
New
0.3
0.4
0.2
1
DegradedDegraded
0
0.2
0.4
Transient
2
Poor
Poor
0
0
0.2
3
Very PoorVery Poor
1
0
0
Steady State
Sum
1.3
0.6
0.8
n-step Probabilities
First Pass
Simulate
Absorbing States

3
Very Poor
0.1
0.4
0.8
0
1.3

Sum
1
1
1
1

Status
Class-1
Class-1
Class-1
Class-1

Economic Data
Type:
Title:

Discount
DTMC EconomicRate per
Road Measure Year
Cost
0.1
0
New
Calculate
1
Degraded
2
Poor
Matrix
3
Very Poor

State Combined
Cost
Cost
0
0
200
200
600
600
2000
12000

[I-alfphaP] =

Inverse[I-alfphaP]

0
New
0
0
0
10000

Transition Cost Matrix

1
2
Degraded Poor
0
0
0
0
0
0
0
0

0.72727 -0.3636 -0.1818

0 0.81818 -0.3636
0
0 0.81818
-0.9091
0
0
4.54171
3.46617
3.67006
4.12882

31947.9
Inverse[I-alfphaP]C 35026.7
37216.5
41043.6

2.01854
2.76274
1.63114
1.83503

1.90639
1.99815
2.76274
1.73309

on Cost Matrix
3
Very Poor
0
0
0
0

-0.0909
-0.3636
-0.7273
1
2.53336
2.77294
2.93605
3.30306

Transient Analysis
Type:
Title:

DTMC
Road

Start
More
Chart
Matrix

Back

Initial
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

Average Cost
0
1
New
Degraded
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519
0.37037 0.18519

3253.13
2
Poor
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519
0.18519

Discounted Cost 31945.5

3
Step
Cum.
Very Poor
Cost
Cost
0.25926 3259.26 260128
0.25926 3259.26 263387
0.25926 3259.26 266647
0.25926 3259.26 269906
0.25926 3259.26 273165
0.25926 3259.26 276424
0.25926 3259.26 279684
0.25926 3259.26 282943
0.25926 3259.26 286202
0.25926 3259.26 289461
0.25926 3259.26 292721
0.25926 3259.26 295980
0.25926 3259.26 299239
0.25926 3259.26 302498
0.25926 3259.26 305758
0.25926 3259.26 309017
0.25926 3259.26 312276
0.25926 3259.26 315535
0.25926 3259.26 318795
0.25926 3259.26 322054
0.25926 3259.26 325313

NPW
31932
31933.4
31934.8
31936
31937
31938
31938.9
31939.7
31940.5
31941.2
31941.8
31942.3
31942.8
31943.3
31943.7
31944.1
31944.5
31944.8
31945.1
31945.3
31945.5

Steady State Analysis

Type: DTMC
Title: Road
Solve by: Inverse

Matrix

Average
0
1
2
3
Cost
New
Degraded Poor
Very Poor per period
Steady State Dist. 0.37037 0.18519 0.18519 0.25926 3259.2593

Markov Chain Transition Matrix

Type:
Title:

DTMC
ExP1

Step
CalculateMeasure
Change
Week
Analyze
State
3
Index Names
Economics
0
State 0 State 0
1
State 1 State 1
Transient
2
State 2 State 2
Sum
Steady State
n-step Probabilities
First Pass
Simulate
Absorbing States

Matrix Analyzed.
3 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
State 0 State 1 State 2
0.8
0
0.2
0.4
0.3
0.3
0
0.9
0.1
1.2
1.2
0.6

Sum

Status
1 Class-1
1 Class-1
1 Class-1

Economic Data
Type:
Title:

Discount
DTMC EconomicRate per
ExP1 Measure Week
Cost
0.1
0
State 0
Calculate
1
State 1
2
State 2
Matrix

State Combined
0
Cost
Cost
State
0
0
0
0
0
0

0
0
0
0

Transition Cost Matrix

1
2
State 1 State 2
0
0
0
0
0
0

on Cost Matrix

Steady State Analysis

Type: DTMC
Title: ExP1
Solve by: Inverse

Matrix

Average
0
1
2
Cost
State 0 State 1 State 2 per period
Steady State Dist. 0.52941 0.26471 0.20588
0

n-Step Transition Matrix

Type:
Title:

DTMC
ExP1
Start
More
Matrix

4 step Transition Matrix

Steps/Iter
1
0
1
2
State 0 State 1 State 2
0
State 0
0.5536 0.2448 0.2016
1
State 1
0.512 0.2808 0.2072
2
State 2
0.4896 0.2952 0.2152

4 step Cost
Average
per step
0
0
0

4 step Cost
Present
Worth
0
0
0

Markov Chain Transition Matrix

Type:
Title:

DTMC
ExP2

Step
CalculateMeasure
Change
Week
Analyze
State
Index Names
Economics
0
State 0
1
State 1
Transient
2
State 2
3
State 3
Steady State
n-step Probabilities
First Pass
Simulate
Absorbing States

4
State 0
State 1
State 2
State 3
Sum

Matrix Analyzed.
2 Recurrent States
1 Recurrent State Class
2 Transient States
0
1
2
State 0 State 1 State 2
0.7
0
0
0.2
0.2
0.4
0.6
0.1
0.1
0.2
0
0
1.7
0.3
0.5

3
State 3
0.3
0.2
0.2
0.8
1.5

Sum
1
1
1
1

Status
Class-1
Transient
Transient
Class-1

Economic Data
Type:
Title:

Discount
DTMC EconomicRate per
ExP2 Measure Week
Cost
0.1
0
State 0
Calculate
1
State 1
2
State 2
Matrix
3
State 3

State Combined
0
Cost
Cost
State
0
0
0
0
0
0
0
0

0
0
0
0
0

Transition Cost Matrix

1
2
State 1 State 2
0
0
0
0
0
0
0
0

on Cost Matrix
3
State 3
0
0
0
0

Steady State Analysis

Type: DTMC
Title: ExP2
Solve by: Inverse

Matrix

Average
0
1
2
3
Cost
State 0 State 1 State 2 State 3 per period
Steady State Dist.
0.4
0
0
0.6
0

n-Step Transition Matrix

Type:
Title:

DTMC
ExP2
Start
More
Matrix

12 step Transition Matrix

Steps/Iter
1
0
1
2
3
State 0 State 1 State 2 State 3
0
State 0 0.40015
0
0 0.59985
1
State 1 0.40038 3E-006 4E-006 0.59961
2
State 2 0.40026 1E-006 2E-006 0.59973
3
State 3
0.3999
0
0 0.6001

12 step Cost
Average
per step
0
0
0
0

Present
Worth
0
0
0
0

Transient Analysis
Type:
Title:

DTMC
ExP2

Start
More
Chart
Matrix

Back

Initial
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

Average Cost
0
1
State 0 State 1
0
0
0.6
0.1
0.54
0.03
0.49
0.011
0.4518 0.0039
0.42834 0.00139
0.41504 0.0005
0.40783 0.00018
0.40402 6E-005
0.40205 2E-005
0.40104 8E-006
0.40052 3E-006
0.40026 1E-006
0.40013 4E-007
0.40007 1E-007
0.40003 5E-008
0.40002 2E-008
0.40001 6E-009
0.4 2E-009
0.4 7E-010
0.4 3E-010

0 Discounted Cost
2
3
Step
State 2 State 3
Cost
1
0
0
0.1
0.2
0
0.05
0.38
0
0.017
0.482
0
0.0061 0.5382
0
0.00217 0.5681
0
0.00077 0.58369
0
0.00028 0.59172
0
1E-004 0.59581
0
3E-005 0.59789
0
1E-005 0.59894
0
4E-006 0.59947
0
2E-006 0.59973
0
6E-007 0.59987
0
2E-007 0.59993
0
7E-008 0.59997
0
3E-008 0.59998
0
9E-009 0.59999
0
3E-009
0.6
0
1E-009
0.6
0
4E-010
0.6
0

0
Cum.
Cost
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

NPW
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0