STATS 200 (Stanford University, Summer 2015)

Lecture 9:

Likelihood Ratio Tests

Until now, we have considered problems where an appropriate test statistic can be chosen
by common sense. However, we may encounter problems in which it is not clear what
test statistic to use. Our first general method for finding tests is based on the likelihood
function LX () on the sets 0 and 1 .
Likelihood Ratio Statistic
Let = 0 1 . The likelihood ratio statistic (X) is defined as
(X) =

sup0 LX ()
,
sup LX ()

and the likelihood ratio test rejects H0 if and only if (X) k, or equivalently, [(X)]1 c,
where k (0, 1) or c = k 1 (0, 1) is chosen to specify the level of the test. Notice from the
definition that 0 (X) 1.
Note: It is possible for the numerator of (X) to be zero while the denominator is
nonzero, but the opposite is not possible. This is one reason why (X) is defined the
way it is (as opposed to the inverse of this quantity, i.e., as opposed to reversing the
numerator and denominator), despite the fact that it leads to a rejection region as a
set where (X) is less than (as opposed to greater than) or equal to something.

Simple Null Hypothesis

The likelihood ratio statistic can be written in a more convenient form if the following two
conditions both hold:
The null hypothesis is simple (H0 = 0 ).
The maximum likelihood estimator of on the parameter space = 0 1 exists.
Then
(X) =

LX (0 )
.

LX ()

Example 9.0.1: Let X1 , . . . , Xn iid Exp(), where > 0, and consider testing H0 = 2
versus H1 2. The likelihood is (for > 0)
n

LX () = n exp( Xi ).
i=1

Earlier in the course, we showed that the maximum likelihood estimator of is

=

n
n
i=1 Xi

= (X ) .

Lecture 9: Likelihood Ratio Tests

Then
n

LX (2) = 2n exp(2 Xi ) = exp[n(2X log 2)],

=(
LX ()

i=1
n

)
n
i=1 Xi

exp(n) = exp[n(1 + log X )].

Then the likelihood ratio statistic is

(X) =

LX (2) exp[n(2X log 2)]

=
= exp[n(1 + log 2 + log X 2X)]

exp[n(1 + log X )]
LX ()
n

= [2X exp(1 2X )] .
Thus, the likelihood ratio test of these hypotheses rejects H0 if and only if
n

[2X exp(1 2X )] k,
or equivalently,

X exp(2X ) c ,

where c = (2e)1 k 1/n . Unfortunately, it is difficult to proceed any further in closed form. We
need the distribution of the test statistic to specify a level via a critical value or to calculate
a p-value, but both (X) itself and the equivalent statistic X exp(2X) are difficult to work
with.

The situation at the end of Example 9.0.1 is not unusual when deriving the form of likelihood
ratio tests. It is often the case that the distribution of the likelihood ratio statistic (or
of some other equivalent statistic) is difficult to actually obtain. This is partially why
people sometimes prefer other approaches to constructing tests. These other approaches are
typically based on asymptotic properties (including asymptotic properties of the likelihood
ratio test), as we will see later.
Composite Null Hypothesis
More work is required to find the likelihood ratio test when the null hypothesis is composite,
particularly if there are one or more parameters with values unspecified by H0 . Finding the
numerator of (X) typically requires first maximizing the likelihood function subject to the
constraints of the null hypothesis, then evaluating the likelihood at this point.
Example 9.0.2: Let X1 , . . . , Xn iid N (, 2 ), where R and 2 > 0 are both unknown,
and consider testing H0 = 0 versus H1 0 for some specified 0 R. The numerator
of the likelihood ratio statistic is sup2 >0 LX (0 , 2 ), and evaluating this requires first finding
the value of 2 that maximizes LX (0 , 2 ), or equivalently, `X (0 , 2 ). Observe that
n

n
1
1 n
2
2
2
2
`
(
,

)
=

+
(X

)
=
0

(Xi 0 ) ,
X
0
i
0
2
2
2
2

2
2( ) i=1
n i=1

Lecture 9: Likelihood Ratio Tests

and it is clear that this value is indeed a maximum. Thus, the value of 2 that maximizes
the expression in the numerator of (X) is

02 =

1 n
2
(Xi 0 ) .
n i=1

Now recall that the (unconstrained) maximum likelihood estimators of and 2 are
1 n
2

= (Xi X ) .
n i=1
2

= X,
Then the likelihood ratio statistic is

(X) =

2
(2
02 )n/2 exp[(2
02 ) ni=1 (Xi 0 ) ]

2
1
) ]
(2
2 )n/2 exp[(2
2 ) ni=1 (Xi
n (X X )2 n/2
n/2

i=1 i
(
02 )n/2 exp(n/2)

2
.
= 2 n/2
= ( 2 ) = n
2

0
(
)
exp(n/2)
i=1 (Xi 0 )

Now observe that

n

(Xi 0 ) = (Xi X + X 0 ) = (Xi X ) + n( X 0 ) + 2( X 0 ) (Xi X )

i=1

i=1

i=1
n

i=1

= (Xi X ) + n( X 0 ) .
i=1

Then
n/2
2
n (X X )2 + n( X )2 n/2
n( X 0 )
i=1 i

(X) =
= 1 +

2
2
n
n

i=1 (Xi X )
i=1 (Xi X )

2 n/2

2 n/2
( X 0 )

[T (X)]

}
,
= {1 +
= 1 +

2
n1

where
T (X) =

X 0

X 0
=
,

2 /(n 1)
S 2 /n

where S 2 = (n 1)1 ni=1 (Xi X )2 is the unbiased sample variance. Note that if H0 is
true (i.e., if = 0 ), then the distribution of T (X) is the distribution of the absolute
value of a Students t random variable with n 1 degrees of freedom. Finally, observe that
rejecting H0 if and only if (X) k is equivalent to rejecting H0 if and only if T (X) c ,
where c = [(n 1)(k 2/n 1)]1/2 . Thus, the likelihood ratio test is simply the conventional
t test, and we can obtain a test with size by rejecting H0 if and only if T (X) exceeds the
appropriate quantile of a Students t distribution with n 1 degrees of freedom.