Markov Analysis

- It is a technique which forecasts probabilities of future

occurrences by analyzing presently known probabilities
- Technique has numerous applications in business
- Application areas are market share analysis, bad debt
prediction, prediction of machine breakdown
- Markov analysis assumes that system starts in an initial
state/condition
- Two competing manufacturers have 40% and 60% market
share respectively as initial state
- In next two months market share will change to 45% and
55% respectively
- Change in market share can be predicted, if system’s
probability of changing from one state to another is known
- These probabilities ,may be collected or estimated and
presented in a matrix
- This matrix is known as matrix of transition probabilities
- Matrix of transition probabilities presents the likelihood that
the system will change from one period to next
- States indicate all possible conditions of a process/system
- A machine can be any one of the two states at a particular
point of time, i.e., working or breakdown
- A process/system may have many states
- In a small town there are three grocery stores
- A resident can be a customer of any one of the three shops
at a particular point of time
- It presents, there are three states corresponding to three
grocery shops
- In Markov Analysis, it is assumed that the states are both
collectively exhaustive and mutually exclusive
- Collectively exhaustive means all possible states of a
system/process can be identified
- Mutually exclusive means that a system can be in only one
state at any point of time.
- It presents that a resident can be a customer of any one of
the three grocery stores at any point of time
- In case of a problem, first possible states are identified
- Probability that the system is in a particular state is
determined
- Vector of state probabilities is formed
- π (i) = vector of state probabilities for period ‘i’
= (π 1, π 2, π 3,……, π n)
- n = number of states
- π 1, π 2, π 3,……, π n = probability of being in state 1, state 2,
….., state n
- If a machine is working in time period 1, the probability of
working state is 1 and probability of breakdown state is 0
- Vector of state probabilities in period 1 for the machine can
be presented as
- π (1) = (1,0)
- π 1 = 1 = probability of being machine is in first state, i.e.,
working
- π 2 = 0 = probability of being machine is in second state, i.e.,
breakdown
Grocery Store Example
- Three grocery store in a small town
- A resident can be a customer of any one grocery store at a
particular point of time
- Population of the town is 1,00,000
- 40,000 people shop at American Food Store, which will be
termed as State 1
- 30,000 people shop from Food Mart, which will be termed as
State 2
- Remaining 30,000 people are shopping from Atlas Foods,
which will be called as State 3
- Probability that a person will be shopping at one of the three
grocery store are as follows:
- State 1 – American Foods Store – (40000/100000)=0.4
- State 2 – Food Mart – (30000/100000)=0.3
- State 3 – Atlas Foods – (30000/100000)=0.3
- Vector of state probabilities during time period 1 for the
grocery shop example is as follows:
- π (1) = (1,0) = (0.4,0.3, 0.3)
- π (1) = Vector of state probabilities during time period 1 for
the grocery shop
- π 1=0.4=probability that a resident will shop at State 1
- π 2=0.3=probability that a resident will shop at State 2
- π 3=0.3=probability that a resident will shop at State 3
- These probabilities also presents the market share of three
grocery store
- After identifying states and state probabilities, matrix of
transitional probabilities is determined
- Matrix of transitional probabilities along with vector of state
probabilities will be used for predicting future events

Matrix of Transition Probabilities

- Matrix of Transition probabilities allow us to move from
current state to future state
- This is a matrix of conditional probabilities of being in a
future state from a current state
- Let, Pij = conditional probability of being in state ‘j’ in future
from state ‘i’ in current period
- P12 = Probability of being in State 2 in future period, given
that the event is in State 1 in preceding period
- P = Matrix of Transition Probabilities

P11 P12 P13…… P1n

P= P21 P22 P23….. P2n
………………………………….
Pm1 Pm2 Pm3 Pmn

- Pij values are determined empirically

- For example, if it has been observed over time that 10% of
the people currently shopping at Store 1 will be shopping at
Store 2 in next period
- This presents P12 = 0.1
- Matrix of Transition probabilities for three grocery shops are
prepared using historical data

0.8 0.1 0.1

P= 0.1 0.7 0.2
0.2 0.2 0.6

- P11 = 0.8 = probability of being in State 1 after being in State

1 in preceding period
- P12 = 0.1 = probability of being in State 2 after being in State
1 in preceding period
- P13 = 0.1 = probability of being in State 3 after being in State
1 in preceding period
- ………………………………………………………………
- P33 = 0.6 = probability of being in State 3 after being in State
3 in preceding period
 Predicting Future Market Share
- State Probabilities for future period may be computed, on
the basis of state probabilities of preceding period and
matrix of transition probabilities
- If current period is 1, state probability for period 2 can be
computed as
- π (2) = π (1) P
- Similarly, state probability for period (n+1) is computed on
the basis of state probabilities of period (n)
- π (n+1) = π (n) P
- Market share for grocery shops for period 2 can be
forecasted as follows:
- π (2) = π (1) P

0.8 0.1 0.1

= (0.4, 0.3, 0.3) 0.1 0.7 0.2
0.2 0.2 0.6

= [(0.4x0.8) + (0.3x0.1) + (0.3x0.2), (0.4x0.1) +

(0.3x0.7) + (0.3x0.2), (0.4x0.1) + (0.3x0.2) + (0.3x0.6)]
= (0.41, 0.31, 0.28)
- It can be observed that American Foods and Food Mart have
improved their market share during period 2
- Market share for Atlas Foods has decreased during period 2
- If this trend continues, does Atlas Foods will lose its total
market share?
- In real life situation if we compute the state probabilities for
number of periods, we will find that the system will come to
equilibrium condition
 Example of Machine Operation
- A company has recorded the operation of its milling machine
for several years
- On the basis of past data, vector of state probabilities and
matrix for transition probabilities have been computed
- π (1) = (1,0)

0.8 0.2
P=
0.1 0.9
- P11 = 0.8 = probability that machine will function correctly this
month given that it was functioning correctly last month
- P12 = 0.2 = probability that machine will not function correctly
this month given that it was functioning correctly last month
- P21 = 0.1 = probability that machine will function correctly this
month given that it was not functioning correctly last month
- P22 = 0.9 = probability that machine will not function correctly
this month given that it was not functioning correctly last
month
- π (2) = π (1) P

0.8 0.2
= (1,0)
0.1 0.9
= (0.8, 0.2)
- π (3) = π (2) P

0.8 0.2
= (0.8, 0.2)
0.1 0.9
= (0.66, 0.34)
 State Probabilities for the Machine
Period State 1 State 2
1 1.0 0.0
2 0.8 0.2
3 0.66 0.34
4 0.562 0.438
5 0.4934 0.5066
6 0.44538 0.55462
7 0.411766 0.588234
8 0.388236 0.611763
9 0.371765 0.628234
10 0.360235 0.639754
11 0.352165 0.647834
12 0.346515 0.653484
13 0.342560 0.657439
14 0.339792 0.660207
15 0.337854 0.662145
- It can be seen that with time the failure rate of the machine
will come to a equilibrium condition