Random Sequences

Gaurav S. Kasbekar
Dept. of Electrical Engineering
IIT Bombay

Recall
We started with one r.v. on a probability space
(, , )
Then, two r.v.s and on a common probability
space (, , )
Then, a vector (1 , , ) of r.v.s on (, , )
Next: an infinite sequence 1 , 2 , 3 , of r.v.s on
(, , )
Well study convergence of such a sequence

Motivation
Two important results have to do with
convergence of random sequences:
1) Law of Large Numbers
2) Central Limit Theorem

Law of Large Numbers

Recall motivation of the definition of ()
independent trials of experiment performed
Average of values of in the trials used to motivate
expression for ()
Let 1 , 2 , 3 , be i.i.d. with mean

1 ++
lim
:

intuitively,

1 ++
,

That is, letting =

the sequence
1 , 2 , 3 , converges to the constant
To state this result formally, need to define
convergence

Central Limit Theorem

1 , 2 , 3 , i.i.d. with mean and variance 2
Informally, for large , the CDF of 1 + +
is approximately Gaussian
That is, letting = 1 + + , the
distribution of converges to a Gaussian
distribution as

Convergence of Real Numbers

1 , 2 , 3 , : a sequence of real numbers

lim = if:

for every > 0, there exists such that

| | < for all

E.g., limit of =

(1)
:

E.g., limit of = (1) :

does not exist (sequence oscillates)

Convergence of Random Variables

1 , 2 , 3 , r.v.s on (, , )
Want to define convergence of this sequence
Recall: is a function from to
So convergence of r.v.s similar to convergence
of functions
Simplest notion: point-wise convergence
called sure convergence in r.v. terminology

Sure Convergence
Definition: 1 , 2 , 3 , converges surely to
if for every , lim () = ()

E.g.: box initially has white and black balls

At each step = 1,2,3, one ball is drawn at
random without replacement (if any left)
: number of white balls left after th draw
Convergence behaviour of 1 , 2 , 3 , :
converges surely to 0

Example
A fair coin tossed an infinite number of times
= 1 if at least one of tosses 1, , results
in heads and = 0 else
Convergence behaviour of 1 , 2 , 3 , :
with probability 1, converges to 1
but for = " " , = 0 for all
does not converge surely to 1

" " is an event of probability 0

Typically we dont care about 0 probability
events

Almost Sure Convergence

Definition: 1 , 2 , 3 , converges almost
surely to if for every ,
lim () = (), where P = 1

In coin tossing example, 1 , 2 , 3 ,

converges a.s. to 1

Example
= 0,1 , = , , = , 0 1
=
For fixed , lim () :

0, 0 < 1

1,
= 1.

1 , 2 , 3 , converges a.s. to:

0

Thus, one way to show a.s. convergence of 1 , 2 , 3 ,

to :
identify such that lim () = () for all

show that P = 1

Almost Sure Convergence

In several examples where intuitively
1 , 2 , 3 , seems to converge to ,
a.s. convergence does not hold

Example
= 0,1 , = , ,

= , 0 1

Ref: Hajek, Chapter 2

Example (contd.)
lim ():

does not exist for any !

So 1 , 2 , 3 , does not converge a.s. to any

r.v.
But intuitively, the sequence seems to be
converging to 0