STAT 410

Convergence and Large Sample

Approximations Part 1

Fall 2016

Definition: Convergence in Distribution

Let { } be a sequence of random variables and let be a random variable. Let

and be, respectively, the cdfs of and . Let ( ) denote the set of all

points where is continuous. We say that converges in distribution to if

lim () = () , ( )

We denote this convergence by

Definition: Convergence in Probability

Let 1 , 2 , be an infinite sequence of random variables, and let be another

random variable. Then the sequence { } converges in probability to , if for all
> 0,
lim P(| | ) = 0, lim P(| | < ) = 1

and write .

Theorem 1

Theorem 2

, constant

Example 1 Consider a sequence of discrete random variables where

( = 0) =

3
1
1
and = = , = 1,2,3,
4
4

0, < 0
1
,
0

<
For each n the cdf is () = 4

1
1,
1

STAT 410

We see that

Convergence and Large Sample

Approximations Part 1

Fall 2016

< 0 implies () 0 as
> 0 implies () 1 as

Therefore () () =

0, < 0
for every 0, i.e. for every point of
1, 0

continuity for F. This is the cdf of the constant 0, i.e. ( = 0) = 1. Thus 0.

By Theorem 2 we would conclude that 0 as well. For a direct proof note that
for any fixed > 0 we have

3
,
4
(| | ) =
0,

1
<

So the probability = 0 eventually, for all > . Since > 0 was arbitrary, we

conclude by definition that 0.

STAT 410

Fall 2016

Convergence and Large Sample

Approximations Part 1
1

Example 2. Let = + where Y and Z are independent (0,1) random

variables. It follows that ~(0, 1 +

). To establish its limiting distribution

consider the limiting moment generating function:

1
1
1+ 2 2

() = 2

1 2

2 = () as

By a result stated below (Theorem 7) this implies that .

We could have also proven this directly via the cdf. Note that

() =

1 +

for all t. Hence, by definition .

() = ()

1
2

In fact we can show a stronger convergence. For any > 0 we have

(| | ) =

= (|| )

= 21 () 0

Therefore by definition we have that , which also imples .

STAT 410

Fall 2016

Convergence and Large Sample

Approximations Part 1

Example 3. Heres an example where convergence in distribution holds, but not

convergence in probability: Let = for all n where ~(0,1). Then

but (| | ) = (|| ) = || = 2 1 > 0 for all n.

2
2
It follows that does not converge in probability to Z.

Example 4.
Let 1 , 2 , be i.i.d. Uniform(0, ). Let = max(1 , 2 , , ).

First show that . This follows because, given any > 0 and less than ,

(| | ) = ( ) =
,

which converges to 0 as n increases, because |( )/| < 1.

Next find the limiting distribution of = ( ).

() = max () = , 0 < < .

() = [( ) ] = > = 1 1 , 0 < < .

() 1 , > 0, .

, where ~().

STAT 410

Fall 2016

Convergence and Large Sample

Approximations Part 1

Example 5. Let 1 , , be a random sample from the distribution with

probability density function
1 1
(;
) = , 0 < < 1,0 < <

Let 1 < 2 < denote the corresponding order statistics.

a)

For which values of does = (1 ) converge in distribution? Find

the limiting distribution of .

() = ( ) = , 0 < < 1

() = (1 ) = 1
= 1 1

/
, 0 < < .

If = 1, lim () = 1 , 0 < < , Then ~().

If < 1, lim () = 1 , 0 < < ,

If > 1, lim () = 0 , 0 < < ,
b)

Then does not have a limiting

distribution.

For which values of does = 1 converge in distribution? Find the

limiting distribution of .

1 () = (1 ) = 1 1

() = 1
If = ,

0 < < ,

1

1

, 0 < < 1

= 1 1 , 0 < < .

lim () = 1

1/

Then 0 and thus 0.

Then ~().

STAT 410

If < ,
If > ,

Convergence and Large Sample

Approximations Part 1

Fall 2016

lim () = 1 , 0 < < , Then

0, and thus 0.

lim () = 0 , 0 < < , Then does not have a limiting

distribution.

, is continuous on the support of

Theorem 3

( ) ()

Theorem 4

, 0 +

Theorem 5

Slutskys Theorem

, ,

+ +

() () for || < .

Theorem 6

Example 6. Let ~ , = . Find the limiting distribution of .

Let ~ , = . Then

() =

() = 1 + 1 as .

, where ~() (Poisson approximation to

Binomial distribution ).

STAT 410

Fall 2016

Convergence and Large Sample

Approximations Part 1

Example 7. Let ~ 2 (). Recall ( ) = and ( ) = 2.

a)

Let = /. Find the limiting distribution of .

Let ~ 2 () and = /. Then,

() =

= = 1 2 as .

Note () = , where ( = 1) = 1 1 1.
b)

Let = ( )/2. Find the limiting distribution of .

() =

2 2

1 2

, < .
2

By Taylor approximation,

2
1
1
= 1 + + 2 + .

So for < ,
2

STAT 410

Fall 2016

Convergence and Large Sample

Approximations Part 1

2
2
1
1

() = 1 + + 2 + 1

= 1
=

1 2

1
+

1
2

2
1
1 +
1

1 2
2

1 2

= 2

as

As , () 2 = (), where ~(0,1) .

STAT 410

Convergence and Large Sample

Approximations Part 1

Fall 2016

Distribution-free convergence of sample averages

Weak Law of Large Numbers
1 , 2 , , are i.i.d. with mean and variance 2 . Then

1
=

=1

Proof: For every fixed > 0 we have, using Markovs inequality,

(| | > ) = (( )2 > 2 )

2
( )2
= 20

as . Therefore by definition of convergence of probability.

Example 8. Let 1 , 2 , , are i.i.d. with mean and variance 2 and finite
fourth moment 4 = ( 4 ). Then, by the weak law of large numbers,

1
2 (12 ) = 2 + 2

=1

Furthermore, using our previous results we can show convergence of the sample
variance:

1
2 =
( )2
1
=

=1

2 2 (1){( 2 + 2 ) 2 } = 2

1
=1

Example 9. Let 1 , , be iid (0,1). Show the following:

9

STAT 410

Convergence and Large Sample

Approximations Part 1

1
a.

b.

c.

d.

e.

f.

2
1 2 1

=1
2
12
2

=1

=1 ln( ) 1

=1

+1
1

>
=1
2
2

Central Limit Theorem

1 , 2 , , are i.i.d. with mean and variance 2 .

( ) (=1 )
=
~(0,1).

10

Fall 2016