Let .; F ; P/ be a probability space and .

Xn / be a sequence of random variables defined on this probability space

taking values countable state space E, satisfying the condition
P.Xn D xn j Xn

D xn

1 ; : : : ; X0

D x0 / D P.Xn D xn j Xn

D xn

for all n  1, x1 ; : : : ; xn 2 E.

1 /;

(1)

Then .Xn / is a (discrete time) Markov process in that

a:s:

P.Xn 2 A j Fs / D P.Xn 2 A j Xs /;

for all n  s  1, and A  E;

(?)

where Fn D  .X0 ; : : : ; Xn / for all n  0.

Notation 1.

P.Xn 2 A j G / for a  -algebra G means E1Xn 2A j G ; P.Xn 2 A j Xs / means E1Xn 2A j  .Xs /.

Notation 2.

EX I F for F  F means EX 1F .

Step 0.

(1) could be extended to

P.Xn 2 A j Xn

D xn

for all n  1, A  E, and x0 ; : : : ; xn

2 E.

1 ; : : : ; X0

D x0 / D P.Xn 2 A j Xn

D xn

1 /;

(2)

Step 1. (?) obvious holds when s D n; we will first establish that (?) holds when s D n 1.
Let H be -system1 consisting of  and all sets of the form fXn 1 D xn 1 ; : : : ; X0 D x0 g. It is clear that
EP.Xn 2 A j Xn
And for H D fXn

D xn

1 /I 

1 ; : : : ; X0

EP.Xn 2 A j Xn

D EE1Xn 2A j Xn

1 I 

D P.Xn D A/ D EP.Xn 2 A/I :

(3)

D x0 g 2 H ,

1 /I H

D EP.Xn 2 A j Xn
D P.Xn 2 A j Xn
D P.Xn 2 A j Xn

1 /I Xn 1

D xn ; : : : ; X 0 D x0

D xn

1 /P.Xn

D xn

1 /P.H /;

where the second equality is due to the fact that P.Xn 2 A j Xn

xn 1 /. But of course,

1/

D xn ; : : : ; X 0 D x0 /

is constant on H , taking value P.Xn 2 A j Xn

E1Xn 2A I H D EE1Xn 2A j Fn I H
D EP.Xn 2 A j Fn /I H
D P.Xn 2 A j Xn

D xn

(4)

(5)
1 ; : : : ; X0 D x0 /P.H /

where the first equality is the defining relation of the conditional expectation, since H 2 H  Fn , and the second
equality is due to P.Xn 2 A j Fn / taking the constant value P.Xn 2 A j Xn 1 D xn 1 ; : : : ; X0 D x0 / on H .
Step 0 allows us to conclude that
EP.Xn 2 A j Xn

1 /I H

D E1Xn 2A I H ;

for all H 2 H :

(6)

But now the Monotone Class Theorem allows us to conclude that

EP.Xn 2 A j Xn
Therefore, P.Xn 2 A j Xn
equal a.s:

1/

1 /I H

D E1Xn 2A I H ;

for all H 2  .H / D Fn

is a version of the conditional expectation P.Xn 2 A j Fn

P.Xn 2 A j Xn

1/

D P.Xn 2 A j Fn

1 /,

1:

and any two versions are

1 /:

1A collection of events is said to be a -system if it is closed under finite intersections; H1 ; H2 2 H H) H1 \ H2 2 H .

(7)

(8)

Step 2.

We will extend (1) to:

P.Xn D xn j Xs D xs ; : : : ; X0 D x0 / D P.Xn D xn j Xs D xs /;

(9)

for all n > s  1, x0 ; : : : ; xs 2 E, xn 2 E.

Fix s, and go by induction on n. There is nothing to prove n D s C 1it is simply (1). Carry through the calculation
X
P.Xn D xn j Xs D xs ; : : : ; X0 / D
P.Xn D xn j X D xn 1 ; Xs D xs ; : : : ; X0 D x0 /P.Xn 1 D x j Xs D xs ; : : : ; X0 D x0 /
x

P.Xn D xn j Xn

D x; Xs D xs /P.Xn

D x j Xs D x s /

D P.Xn D xn j Xs D xs /
(10)
in which
P.Xn D xn j Xn

D xn

1 ; Xs

D xs ; : : : ; X0 D x0 / D P.Xn D xn j Xn

D xn

1 ; Xs

D xs /;

(11)

by Step 1 as both are equal to the common value P.Xn D xn j Xn 1 D xn 1 /; and P.Xn 1 D xjXs D xs ; : : : ; X0 D
x0 / D P.Xn 1 D x j Xs D xs / by the induction hypothesis. The summation is taken over all such that P.Xn 1 D
x; Xs D s; : : : ; X0 D x0 / > 0.
Now we can extend this result to
P.Xn 2 A j Xs D xs ; : : : ; X0 D x0 / D P.Xn 2 A j Xs D xs /:
Step 3.
step:

(12)

Just as Step 0 provided for the result in Step 1, Step 2 (i.e. (12)) provides the essential ingredient in this final
P.Xn 2 A j Fs / D P.Xn 2 A j Xs / a.s.

(?)

for all A  E, n  s  1. The proof is exactly the same as Step 1.

In particular, (?) says that for all n  s  1, x 2 E and A0 ; : : : ; As
P.Xn 2 A j Xs D x; Xs

2 As

1 ; : : : ; X0

1; A

 E,

2 A0 / D P.Xn 2 A j Xs D x/:

(13)