Professional Documents
Culture Documents
Chia-Ping Chen
Professor
Department of Computer Science and Engineering
National Sun Yat-sen University
Probability
Dependence between Random Variables
X is independent of Y given Z if
X
Y |Z
S0 , S1 , . . .
{Sn+1 , Sn+2 , . . . }
{Sn1 , . . . , S0 } | Sn , nN
S0 = s0 , . . . , Sn = sn
is
P(S0 = s0 , . . . , Sn = sn )
= P(S0 = s0 )P(S1 = s1 |S0 = s0 ) . . . P(Sn = sn |Sn1 = sn1 , . . . )
= P(S0 = s0 )P(S1 = s1 |S0 = s0 ) . . . P(Sn = sn |Sn1 = sn1 )
" n #
Y
= s0 psk1 sk
k=1
where
i = P(S0 = i)
is the probability of an initial state.
R(n) = Tn
By definition
R(1) = T
By induction
R(n) = Tn
Trellis
an expansion of the state transition graph over time
i j
i j j i
for every j.
C = S1 Sd
so that all valid state transitions are from one subset to the
next subset cyclicly
S1 S2 Sd S1
For a large n
m
X
P(Xn = j) j P(X0 = i) = j
i=1
Let n m
X
j = k pkj
k=1
and sum to 1
1 = 1 + 2
0 = 2 (1 p)
1 = 1 (1 p) + 2 p
2 = 0 1 + 1 p
1 = 0 + 1 + 2
Let vij (n) be the number of visits to state j within the first n
transitions, starting from state i. Then
vij (n) a. s.
j
n
X
j = k pkj
k
P(Sn+1 = i + 1|Sn = i) = bi
P(Sn+1 = i 1|Sn = i) = di
i1 bi1 = i di , i = 2, . . . , m
i1 bi1 = i di
bi1 b1 . . . bi1
i = i1 = = 1
di d2 . . . di
b0 = = bm1 = b, bm = 0
d0 = 0, d1 = = dm = d
i1 b = i d, i = 1, . . . , m
lead to
b
i = i1 = = i 0 , =
d
m
P
Adding normalization condition i = 1, we get
i=0
1 1 1 i
0 = = and i = 0 i =
1 + + + m 1 m+1 1 m+1
pss = 1
ai = is
ai = P lim Sn = s|S0 = i
n
m
X
= P lim Sn = s, S1 = j|S0 = i
n
j=1
Xm
= P (S1 = j|S0 = i) P lim Sn = s|S1 = j
n
j=1
Xm
= P (S1 = j|S0 = i) P lim Sn1 = s|S0 = j
n
j=1
Xm
= pij aj
j=1
or
(1 p)(ai ai1 ) = p(ai+1 ai )
Defining
1p
i = ai+1 ai , =
p
we have
i = i1
which leads to
i = i1 = = i 0
am = 1, a0 = 0
and
am a0 = (am am1 ) + + (a1 a0 )
= m1 + + 0
= 0 (m1 + + 1)
=1
so
1
0 =
m1 + + 1
Thus
i1 + + 0
ai = a0 + 0 + + i1 =
m1 + + 1
i = E[A|S0 = i]
where
A = arg min {Sn is recurrent}
n>0
1 = 0
2 = 1 + 0.31 + 0.42 + 0.33
3 = 1 + 0.32 + 0.43 + 0.34
4 = 0
Thus
10
2 = 3 =
3
Note that
j 6= i
When in state i
a fraction pij of the state transitions out of i go to state j
on average, the number of state transitions from i to j per
unit time is
i pij
the rate of transitions from state i to state j is
qij = i pij
1 = 1, 2 = 5, 3 = 3
qij , i 6= j
Z = Z0 , Z1 , . . .
where
Zn = X(n)
is an auxiliary discrete-time Markov chain of X(t).
For j 6= i
pij () = P(Zn+1 = j|Zn = i)
transition out of i transition to j
z }| {
= (i + o())
z}|{
pij
= i pij + o()
= qij + o()
For j = i X
pii () = 1 pij ()
j6=i
X
=1 qij + o()
j6=i
For a CTMC X(t) with single recurrent class, each state has a
steady-state probability
lim P(X(t) = j) = j
t
30
1 =
41
6
2 =
41
5
3 =
41
That is
qij = 0, for |i j| > 1
Normalization
0 + 1 + + m = 1 0 (1 + + + m ) = 1
0 = (1 + + + m )1
i
i =
1 + + + m