Markov Chains

Markov Analysis enables to

predict the future state of any
system with the help of state
transition probabilities matrix
Examples - Systems
 Market place for a product and its
competitive brand.
 Machines used to manufacture a
product.
 Billing, Credit and collection procedures
involved in converting accounts
receivable from the product’s sales into
cash.
Markov Chain
A Markov chain includes
A set of states
 A set of associated transition probabilities
 For every pair of states s and s’ (not necessarily
distinct) we have an associated transition
probability T(ss’) of moving from state s to state
s’
 For any time t, T(ss’) is the probability of the

Markov process being in state s’ at time t+1 given

that it is in state s at time t
Characteristics
 Finite number of possible states.
 States are both collectively exhaustive and mutually
exclusive.
 Transition probabilities depend only on the current
state of the system..
 Long run probability of being in a particular state will
be constant over time.
 Transition probabilities of moving to alternative states
in the next time period given a state in the current
time period must sum to 1.0
Markov Process
Simple Example
Weather:
• raining today 40% rain tomorrow
60% no rain tomorrow

• not raining today 20% rain tomorrow

80% no rain tomorrow
Markov Process
Simple Example
Weather:
• raining today 40% rain tomorrow
60% no rain tomorrow

• not raining today 20% rain tomorrow

80% no rain tomorrow
The transition matrix:
• Stochastic matrix:
 0.4 0.6  Rows sum up to 1
P    • Double stochastic matrix:
 0.2 0.8  Rows and columns sum up to 1
State and Transition probabilities
 State Prob – of an event is the
probability of its occurrence at a point in
time.
 Transition Prob – represents the
conditional prob. that a system will be in
a future state based an existing state.
Multi period Transition Probabilities
 R1 = Ro * P
 R1 = state of probability at time n=1
 Ro = state of probability at time n=2

 R2 = R1 * P
Example
 The price of an equity share of a company may increase, decrease or remain constant on
any given day. It is assumed that the change in price on any day affects the change on
the following day as described by the following transition matrix:
 If the price of shares increase today, what are the chances that it will increase, decrease
or remain unchanged tomorrow.
 If the prices of shares decrease today what are the chances that it will increase tomorrow

Changes Changes Tomorrow

Today
Increase Decrease Unchanged

Increase 0.5 0.2 0.3

Decrease 0.7 0.1 0.2

Unchanged 0.4 0.5 0.1

Steps to construct Matrix of Transition
Probabilities
1. Determine retention probabilities
2. Determine gains and losses
probabilities
3. Develop matrix of transition
probabilities
Example-1
 There are 3 dairies A, b and C in a small town which supply all the
milk consumed in the town. Assume that the initial consumer
sample is composed of 1000 respondents distributed over 3
dairies A, B and C. It is known by all the dairies that consumers
switch from one dairy to another due to advertising, price and
dissatisfaction. All these dairies maintain records of the number of
their customers and dairy from which they obtained each new
customer. Assume that the matrix of transition probabilities remain
fairly stable and a t the beginning of period one, market shares are
A= 25%, B= 45% and C= 30 %. Below table shows flow of
customers for an observation period of one month and customers
lost or gained by dairies. Construct the state transition probability
matrix to analyze the problem.

Dairy Period 1 Change during period Period 2

Customers Customers
Gain Loss
A 250 62 50 262
B 450 53 60 443
C 300 50 55 295
1000 165 165 1000
Dairy Period 1 Gains From Losses To Period 2
Customers Customers

A B C A B C

A 250 0 35 27 0 25 25 262

B 450 25 0 28 35 0 25 443

C 300 25 25 0 27 28 0 295
Steady State (Equilibrium)
Conditions
 Itexists when the state probabilities for
a future period are the same as the
state probabilities for a previous period
 Procedure
 Formulate a state transition matrix
 Calculate future probable market share

 Determine steady state condition

 There are 3 dairies A, b and C in a small town which supply all the milk
consumed in the town. It is known by all the dairies that consumers switch
from one dairy to another due to advertising, price and dissatisfaction. All
these dairies maintain records of the number of their customers and dairy from
which they obtained each new customer. Assume that the matrix of transition
probabilities remain fairly stable and a t the July 1, market shares are A= 22%,
B= 49% and C= 29 %. Below table shows flow of customers for an observation
period of one month. Managers of these dairies are willing to know
 Market share of their dairies on 1st August and 1st September
 Market shares in steady state

Dairy June1 Gains From Losses To Period 2

Customers Customers
A B C A B C

A 200 0 35 25 0 20 20 220

B 500 20 0 20 35 0 15 490

C 300 20 15 0 25 20 0 290