1/27

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Department of Economics
University of Essex

22/24 November 2011

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

2/27

Outline

Two Types of Error

An Alternative Approach

Summary of Testing Procedure

The Power of a Test

Reference: R. L. Thomas, Using Statistics in Economics,

McGraw-Hill, 2005, sections 4.24.4.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

3/27

So far we have adopted a test criterion of the kind

reject H0 if |TS| > Z 0 ,
where Z 0 depended on the level of significance and was
taken from the standard Normal table.
In this setup, rejection of H0 implied acceptance of HA .
But what if we are unable to reject H0 ?
Do we simply accept H0 and reject HA ?
Suppose we extend our test criterion to:
reject H0 if |TS| > Z 0 but accept H0 if |TS| < Z 0 .
With this criterion:
rejection of H0 implies acceptance of HA ;
acceptance of H0 implies rejection of HA .
EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

4/27

There are two types of error associated with this test

criterion:
Type I Error: we reject H0 when it is true;
Type II Error: we accept H0 when it is false.

Note that type I error is the significance level .

Type II error is usually denoted (the Greek letter beta).
The two types of error are illustrated below:

Reject H0
Accept H0

H0 is true
Type I error

H0 is false

Type II error

Ideally we would like to find a value of Z 0 such that the

probabilities of making type I and type II errors are as small
as possible.
EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

5/27

To explain whats happening we shall return to the annual

incomes example.
The null and alternative hypotheses are (respectively):
H0 : = 17, 670, HA : 6= 17, 670,

This is a two-tail test; the test statistic is:

TS =

17, 670
X

,
/ n

is the sample mean, n is the sample size, and is

where X
the population standard deviation.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

6/27

Using a 0.05 level of significance, the test criterion is:

reject H0 if |TS| > 1.96but accept H0 if |TS| < 1.96.
What is the type I error?
Under H0 , TS N(0, 1); hence
Pr(type I error) = Pr(reject H0 |H0 is true)
= Pr(|TS| > 1.96|TS N(0, 1))
= Pr(|Z| > 1.96) = 0.05,
where Z denotes a standard normal variable.
This confirms that the probability of a type I error is the
level of significance, .

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

7/27

The type II error is more difficult to handle.

When H0 is false, we dont know the value of ; all we
know is that 6= 17, 670.
Suppose we let the unknown value of = 6= 17, 670.
From the Central Limit Theorem,


2

XN ,
,
n
and it follows that (by standardisation)

X
N(0, 1).
/ n

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

8/27

Hence we find that

Pr(type II error) = Pr(accept H0 |H0 is false)
= Pr(1.96 < TS < 1.96| = )





17, 670
X
X


= Pr 1.96 <
< 1.96  N(0, 1) .
/ n
/ n
In order to use this expression we need to write it in terms
of the standard normal variable

X
.
/ n
How can we do this?
EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

9/27

First, add 17, 670/(/ n) to all terms to the left of the

conditioning sign (|):

17, 670
X
17, 670
<
< 1.96 +

Pr 1.96 +
/ n
/ n
/ n

X
N(0, 1) .
/ n

Next, subtract /(/ n) from the same terms:

17, 670
17, 670
X

< 1.96 +

Pr 1.96 +
<
|. . . .
/ n
/ n
/ n

This is now a probability statement about a standard

normal variable; hence
Pr(type II error)

17, 670
17, 670

< Z < 1.96 +

.
= Pr 1.96 +
/ n
/ n

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

10/27

The problem remains that we dont know .

But we could consider some hypothetical values of that
might give us some idea as to these probabilities when
= differs from the value under the null hypothesis
(17,670).
Forexample, suppose that n = 400 and = 1700; then
/ n = 85 so that
Pr(type II error)

17, 670
17, 670
= Pr 1.96 +
< Z < 1.96 +
.
85
85

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

11/27

What values shall we choose for ?

Lets begin by choosing a value of that is some way
from the null value, say = 18, 500.
Substituting the value of we find that
Pr(type II error) = Pr(11.72 < Z < 7.80) = 0
because the values 11.72 and 7.80 are so far out in the
left tail of the N(0, 1) distribution.
This is good because there is virtually no chance of
accepting H0 when it is false and = 18, 500.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

12/27

However, what if is closer to the null value, say

= 17, 750?
In this case
Pr(type II error) = Pr(2.90 < Z < 1.02)
= Pr(0 < Z < 2.90) + Pr(0 < Z < 1.02)
= 0.4981 + 0.3461 = 0.8442.

Here there is a huge probability of accepting H0 when it is

false, which is not so good!

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

13/27

But the problem may not be as bad as it seems.

When is close to the null value, the test is unable to
discriminate between the two outcomes with a high
probability.
But this may not matter too much in this case, because
17,750 is not that far away from 17,670.
However, when is further away from the null value of ,
the probability of type II error falls i.e. the test is able to
discriminate between the null and alternative values.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Two Types of Error

14/27

It can be useful to consider a range of values for and the

probabilities of accepting H0 :

Pr(type II error)

17,000
0

17,250
0.001

17,450
0.265

17,550
0.708

Pr(type II error)

17,750
0.844

17,850
0.437

17,950
0.091

18,500
0

17,670
0.950

When = 17, 670 the reported value (0.95) is equal to

1 Pr(type I error).
When 6= 17, 670 the reported values are Pr(type II error);
these decline as we move away from the null value.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

An Alternative Approach

15/27

If we were to increase the significance level to, say, 10%,

the value at = 17, 670 would fall to 0.90, and the other
values would fall as well.
This implies that increasing Pr(type I error) reduces Pr(type
II error) there is a trade-off between the two probabilities.
Due to the difficulties in handling type II errors, many
statisticians use a slightly different test criterion:
reject H0 if |TS| > Z 0 but reserve judgment if |TS| < Z 0 .
Here, because we do not accept H0 , we cannot make a
type II error!

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

An Alternative Approach

16/27

A pertinent example is provided by jury trials in England

and Scotland, where H0 corresponds to the defendant
being not guilty.
In England (and Wales), a jury has a choice between
finding the defendant guilty (i.e. reject H0 ) or not guilty
(i.e. accept H0 ).
In Scotland, however, a jury has a third option of finding
the case not proven i.e. reserve judgment on H0 .
This is where the probability of finding the defendant
innocent when he is guilty is thought to be non-negligible.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary of Testing Procedure

17/27

There is a six-step procedure for conducting large-sample

tests for which forms the basis for all hypothesis tests.
Step I. State the null and alternative hypotheses.
The null usually contains an equality:
H0 : = q
where q is an appropriate constant.
The alternative for a two-tail test is
HA : 6= q.
The alternative for a one-tail test is
either HA : < q or HA : > q.
Important: the null is always defined in terms of the

population parameter (), not the sample version (X).

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary of Testing Procedure

18/27

Step II. Construct an appropriate test statistic.

This uses the null value of ; for example
TS =

q
X
N(0, 1)
/ n

under H0 .

Step III. Decide on the significance level.

This is the permitted level of the probability of a type I error.
The most common choice is 5%; other frequently used
choices are 10% and 1%.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary of Testing Procedure

19/27

Step IV. Formulate a test criterion.

For a two-tail test this will be of the form:
reject H0 if |TS| > Z 0 but reserve judgment if |TS| < Z 0 .
For an upper-sided one-tail test this will be of the form:
reject H0 if TS > Z 0 but reserve judgment if TS < Z 0 .
For a lower-sided one-tail test this will be of the form:
reject H0 if TS < Z 0 but reserve judgment if TS > Z 0 .
The value of Z 0 , taken from the standard normal table,
depends on the significance level and whether a one- or
two-tail test is performed.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary of Testing Procedure

20/27

Step V. Use the sample information.

and sample standard
Calculate the sample mean, X,
deviation, s, the latter to be used as an estimate of .
Do not change any of the steps IIV above!

Step VI. Compute the value of TS.

And apply the test criterion in step IV.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary of Testing Procedure

21/27

Example (Thomas, Example 4.9). A train company

claims that an inter-city service is on average only 2
minutes late in arriving at a destination. A random sample
of 40 trains on this route turn out to be 3 minutes late on
average with a standard deviation of 3.5 minutes. Test the
companys claim.
Solution. Proceeding with the six steps:
Step I. State the null and alternative hypotheses.
H0 : = 2, HA : > 2.
The company claims the trains are on average only 2
minutes late, suggesting that our alternative hypothesis
should be that they are greater than 2 minutes late.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary of Testing Procedure

22/27

Step II. Construct an appropriate test statistic.

This uses the null value of = 2 i.e.
TS =

2
X
N(0, 1) under H0 .
/ n

Step III. Decide on the significance level.

Lets carry out the test at the 5% significance level i.e. we
are prepared that there is a 5% chance that we will reject
the null hypothesis even if it is true.

Step IV. Formulate a test criterion.

For our upper-sided one-tail test this will be of the form:
reject H0 if TS > Z 0 but reserve judgment if TS < Z 0 .
The value of Z 0 = 1.64.
EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary of Testing Procedure

23/27

Step V. Use the sample information.

= 3 and s = 3.5.
We have n = 40, X

Step VI. Compute the value of TS.

From the sample information we have
TS =

32
= 1.81.
3.5/ 40

As TS > 1.64 we reject H0 : = 2 in favour of HA : > 2 at

the 5% level.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

The Power of a Test

24/27

When we choose the significance level of a test we are

setting the probability of type I error i.e. the probability of
rejecting H0 when it is true.
Typically we would like this to be quite small, hence a 5%
significance level is common.
It is also desirable to have a high probability of rejecting H0
in favour of HA when H0 is false.
But it can difficult to have this as high as we would wish.
This is the power of the test:
power of test = Pr(reject H0 |H0 false).

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

The Power of a Test

25/27

The table on slide 13 gave Pr(accept H0 |H0 false) we can

use these probabilities to obtain
Pr(reject H0 |H0 false) = 1 Pr(accept H0 |H0 false).

We obtain the following for Pr(reject H0 |H0 false):

Power

17,000
1

17,250
0.999

17,450
0.735

17,550
0.292

Power

17,750
0.156

17,850
0.563

17,950
0.909

18,500
1

17,670

Note that, when = 17, 670, H0 is true.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

The Power of a Test

26/27

We can see that Pr(reject H0 |H0 false) increases as

moves away from 17,670.
It is more difficult for the test to reject H0 when false when
is close to the null value than it is when is far away
from the null value.
Put another way, low power arises only when H0 is slightly
incorrect.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I

Summary

27/27

Summary

Type I and type II error.

A six-step testing procedure.
The power of a test.

Next week:
Hypothesis testing in small samples.

EC114 Introduction to Quantitative Economics

7. Hypothesis Testing I