Use of moment generating

functions
Definition
Let X denote a random variable with probability
density function f(x) if continuous (probability mass
function p(x) if discrete)
Then
m
X
(t) = the moment generating function of X
( )
tX
E e =
( )
( )
if is continuous
if is discrete
tx
tx
x
e f x dx X
e p x X

The distribution of a random variable X is

described by either
1. The density function f(x) if X continuous (probability
mass function p(x) if X discrete), or
2. The cumulative distribution function F(x), or
3. The moment generating function m
X
(t)


Properties
1. m
X
(0) = 1
( )
( ) ( )
0 derivative of at 0.
k
th
X X
m k m t t = =
2.
( )
k
k
E X = =
( )
2 3
3 2
1
1 .
2! 3! !
k
k
X
m t t t t t
k

= + + + + + + 3.
( )
( )
( )
continuous
discrete
k
k
k
k
x f x dx X
E X
x p x X

= =

4. Let X be a random variable with moment

generating function m
X
(t). Let Y = bX + a
Then m
Y
(t) = m
bX + a
(t)
= E(e
[bX + a]t
) = e
at
m
X
(bt)
5. Let X and Y be two independent random
variables with moment generating function
m
X
(t) and m
Y
(t) .
Then m
X+Y
(t) = m
X
(t) m
Y
(t)
6. Let X and Y be two random variables with
moment generating function m
X
(t) and m
Y
(t)
and two distribution functions F
X
(x) and
F
Y
(y) respectively.
Let m
X
(t) = m
Y
(t) then F
X
(x) = F
Y
(x).
This ensures that the distribution of a random
variable can be identified by its moment
generating function
M. G. F.s - Continuous distributions

Name
Moment generating
function M
X
(t)
Continuous
Uniform
e
bt
-e
at
[b-a]t

Exponential
t



(

(
for t <
Gamma

t



(

(
o
for t <
_
2

v d.f.

(
(
1
1-2t

v/2
for t < 1/2
Normal
e
t+(1/2)t
2
o
2


M. G. F.s - Discrete distributions


Name
Moment
generating
function M
X
(t)
Discrete
Uniform
e
t
N

e
tN
-1
e
t
-1

Bernoulli q + pe
t

Binomial (q + pe
t
)
N

Geometric pe
t
1-qe
t

Negative
Binomial

(
(
pe
t
1-qe
t

k

Poisson
e
(e
t
-1)


Moment generating function of the
gamma distribution
( )
( )
( )
tX tx
X
m t E e e f x dx

= =
}
( ) ( )
1
0
0 0
x
x e x
f x
x
o
o

>

I =

<

where
( )
( )
( )
tX tx
X
m t E e e f x dx

= =
}
( )
1
0
tx x
e x e dx
o
o

=
I
}
using
( )
( ) 1
0
t x
x e dx
o

=
I
}
( )
1
0
1
a
a bx
b
x e dx
a

=
I
}
( )
1
0
a bx
a
a
x e dx
b

I
=
}
or
then
( )
( )
( ) 1
0
t x
X
m t x e dx
o

=
I
}
( )
( )
( )
t
o
o
o

o

I
=
I

t
t
o

| |
= <
|

\ .
Moment generating function of the
Standard Normal distribution
( )
( )
( )
tX tx
X
m t E e e f x dx

= =
}
( )
2
2
1
2
x
f x e
t

=
where
thus
( )
2 2
2 2
1 1
2 2
x x
tx
tx
X
m t e e dx e dx
t t

+

= =
} }
We will use
( )
2
2
2
0
1
1
2
x a
b
e dx
b t

=
}
( )
2
2
1
2
x
tx
X
m t e dx
t

=
}
2
2
2
1
2
x tx
e dx
t

=
}
( )
2
2 2 2 2
2
2 2 2 2
1 1
2 2
x t
x tx t t t
e e dx e e dx
t t
+


= =
} }
2
2
t
e =
Note:
( )
2
2 3
2 2
2
2
2 2
1
2 2! 3!
t
X
t t
t
m t e
| | | |
| |
\ . \ .
= = + + + +
2 3 4
1
2! 3! 4!
x
x x x
e x = + + + + +
2 4 6 2
2 3
1
2 2 2! 2 3! 2 !
m
m
t t t t
m
= + + + + + +
Also
( )
2 3
3 2
1
1
2! 3!
X
m t t t t

= + + + +
Note:
( )
2
2 3
2 2
2
2
2 2
1
2 2! 3!
t
X
t t
t
m t e
| | | |
| |
\ . \ .
= = + + + +
2 3 4
1
2! 3! 4!
x
x x x
e x = + + + + +
2 4 6 2
2 3
1
2 2 2! 2 3! 2 !
m
m
t t t t
m
= + + + + + +
Also
( )
2 3
3 2
1
1
2! 3!
X
m t t t t

= + + + +
( )
moment
th k
k
k x f x dx

= =
}
Equating coefficients of t
k
, we get
( )
2
1
for 2 then
2 ! 2 !
m
m
k m
m m

= =
0 if is odd and
k
k =
1 2 3 4
hence 0, 1, 0, 3 = = = =
Using of moment generating
functions to find the distribution of
functions of Random Variables
Example
Suppose that X has a normal distribution with
mean and standard deviation o.
Find the distribution of Y = aX + b
( )
2 2
2
t
t
X
m t e
o
+
=
Solution:
( ) ( )
( )
( )
2
2
2
at
at
bt bt
aX b X
m t e m at e e
o
+
+
= =
( )
2 2 2
2
a t
a b t
e
o
+ +
=
= the moment generating function of the normal
distribution with mean a + b and variance a
2
o
2
.

Thus Z has a standard normal distribution .
Special Case: the z transformation
1 X
Z X aX b

o o o

| | | |
= = + = +
| |
\ . \ .
1
0
Z
a b


o o

| | | |
= + = + =
| |
\ . \ .
2
2 2 2 2
1
1
Z
a o o o
o
| |
= = =
|
\ .
Thus Y = aX + b has a normal distribution with
mean a + b and variance a
2
o
2
.
Example
Suppose that X and Y are independent each having a
normal distribution with means
X
and
Y
, standard
deviations o
X
and o
Y

Find the distribution of S = X + Y
( )
2 2
2
X
X
t
t
X
m t e
o
+
=
Solution:
( )
2 2
2
Y
Y
t
t
Y
m t e
o
+
=
( ) ( ) ( )
2 2 2 2
2 2
X Y
X Y
t t
t t
X Y X Y
m t m t m t e e
o o
+ +
+
= =
Now
or
( )
( )
( )
2 2 2
2
X Y
X X
t
t
X Y
m t e
o o

+
+ +
+
=
= the moment generating function of the
normal distribution with mean
X
+
Y
and
variance

2 2
X Y
o o +
Thus Y = X + Y has a normal distribution
with mean
X
+
Y
and variance

2 2
X Y
o o +
Example
Suppose that X and Y are independent each having a
normal distribution with means
X
and
Y
, standard
deviations o
X
and o
Y

Find the distribution of L = aX + bY
( )
2 2
2
X
X
t
t
X
m t e
o
+
=
Solution:
( )
2 2
2
Y
Y
t
t
Y
m t e
o
+
=
( ) ( ) ( ) ( ) ( )
aX bY aX bY X Y
m t m t m t m at m bt
+
= =
Now
( )
( )
( )
( )
2 2
2 2
2 2
X Y
X Y
at bt
at bt
e e
o o
+ +
=
or
( )
( )
( )
2 2 2 2 2
2
X Y
X X
a b t
a b t
aX bY
m t e
o o

+
+ +
+
=
= the moment generating function of the
normal distribution with mean a
X
+ b
Y

and variance

2 2 2 2
X Y
a b o o +
Thus Y = aX + bY has a normal
distribution with mean a
X
+ B
Y
and
variance

2 2 2 2
X Y
a b o o +
Special Case:
Thus Y = X - Y has a normal distribution
with mean
X
-
Y
and variance

( ) ( )
2 2
2 2 2 2
1 1
X Y X Y
o o o o + + = +
a = +1 and b = -1.

Example (Extension to n independent RVs)
Suppose that X
1
, X
2
, , X
n
are independent each having a
normal distribution with means
i
, standard deviations o
i
(for i = 1, 2, , n)
Find the distribution of L = a
1
X
1
+ a
1
X
2
+ + a
n
X
n

( )
2 2
2
i
i
i
t
t
X
m t e
o
+
=
Solution:
( ) ( ) ( )
1 1 1 1 n n n n
a X a X a X a X
m t m t m t
+ +
= Now
( )
( )
( )
( )
2 2
2 2
1 1
1 1
2 2
n n
n n
a t a t
a t a t
e e
o o
+ +
=
(for i = 1, 2, , n)
( ) ( )
1
1
n
X X n
m at m a t =
or
( )
( )
( )
2 2 2 2 2
1 1
1 1
1 1
...
...
2
n n
n n
n n
a a t
a a t
a X a X
m t e
o o

+ +
+ + +
+ +
=
= the moment generating function of the
normal distribution with mean
and variance

Thus Y = a
1
X
1
+ + a
n
X
n
has a normal
distribution with mean a
1

1
+ + a
n

n

and variance

1 1
...
n n
a a + +
2 2 2 2
1 1
...
n n
a a o o + +
2 2 2 2
1 1
...
n n
a a o o + +


1 2
1
n
a a a
n
= = = =
1 2 n
= = = =
2 2 2 2
1 1 1
o o o o = = = =
In this case X
1
, X
2
, , X
n
is a sample from a
normal distribution with mean , and standard
deviations o, and
( )
1 2
1
n
L X X X
n
= + + +
the sample mean X = =
Special case:
Thus
2 2 2 2 2
1 1
...
x n n
a a o o o = + +
and variance

1 1
...
x n n
a a = + +
has a normal distribution with mean
1 1
...
n n
Y x a x a x = = + +
( ) ( )
1
1 1
...
n
x x
n n
= + +
( ) ( )
1 1
...
n n
= + + =
2 2 2
2
2 2 2
1 1 1
... n
n n n n
o
o o o
| | | | | |
= + + = =
| | |
\ . \ . \ .


If x
1
, x
2
, , x
n
is a sample from a normal
distribution with mean , and standard
deviations o, then
the sample mean x =
Summary
2
2
x
n
o
o =
and variance

x
=
has a normal distribution with mean
standard deviation
x
n
o
o
| |
=
|
\ .
0
0.1
0.2
0.3
0.4
20 30 40 50 60
Population
Sampling distribution
of
x


If x
1
, x
2
, , x
n
is a sample from a distribution
with mean , and standard deviations o, then
if n is large
the sample mean x =
The Central Limit theorem
2
2
x
n
o
o =
and variance

x
=
has a normal distribution with mean
standard deviation
x
n
o
o
| |
=
|
\ .


We will use the following fact:
Let
m
1
(t), m
2
(t),
denote a sequence of moment generating functions
corresponding to the sequence of distribution
functions:
F
1
(x) , F
2
(x),
Let m(t) be a moment generating function
corresponding to the distribution function F(x) then
if
Proof: (use moment generating functions)
( ) ( )
lim for all in an interval about 0.
i
i
m t m t t

=
( ) ( )
lim for all .
i
i
F x F x x

=
then


Let x
1
, x
2
, denote a sequence of independent
random variables coming from a distribution with
moment generating function m(t) and distribution
function F(x).
( ) ( ) ( ) ( ) ( )
1 2 1 2
=
n n n
S x x x x x x
m t m t m t m t m t
+ + +
=
Let S
n
= x
1
+ x
2
+ + x
n
then
( )
=
n
m t (

1 2
now
n n
x x x S
x
n n
+ + +
= =
( ) ( )
1
or
n
n
n
x S
S
n
t t
m t m t m m
n n
| |
|
\ .
(
| | | |
= = =
| | (
\ . \ .



Let
x n n
z x
n

o
o o

= =
( )
then
n
n n
t t
z x
nt nt
m t e m e m
n

o o
o o

(
| | | |
= =
( | |
| |
(
\ . \ .

( )
and ln ln
z
n t
m t t n m
n

o
o
(
| |
( = +
( |

\ .



( )
Then ln ln
z
n t
m t t n m
n

o
o
(
| |
( = +
( |

\ .

( )
2 2
2 2 2
ln
t t
m u
u u

o o
( = +

2
2 2
Let or and
t t t
u n n
u u
n
o o
o
= = =
( )
2
2 2
ln m u u
t
u

o
(

=


( )
( )
( )
( )
0
Now lim ln lim ln
z z
n u
m t m t

( ( =

( )
2
2 2
0
ln
lim
u
m u u
t
u

o

(

=
( )
( )
2
2
0
lim using L'Hopital's rule
2
u
m u
m u
t
u

o

'

=
( ) ( ) ( )
( )
2
2
2
2
0
lim using L'Hopital's rule again
2
u
m u m u m u
m u
t
o

'' ' (

(

=


( ) ( ) ( )
( )
2
2
2
2
0
lim using L'Hopital's rule again
2
u
m u m u m u
m u
t
o

'' ' (

(

=
( ) ( )
2
2
2
0 0
2
m m
t
o
'' ' (

=
( )
( )
2
2
2 2
2
2 2
i i
E x E x
t t
o
(

= =
( )
( )
( ) ( )
2
2
2
thus lim ln and lim
2
t
z z
n n
t
m t m t e

( = =



( )
2
2
Now
t
m t e =
Is the moment generating function of the standard
normal distribution
Thus the limiting distribution of z is the standard
normal distribution
( )
2
2
1
i.e. lim
2
x u
z
n
F x e du
t

=
}
Q.E.D.