Professional Documents
Culture Documents
13.1 Introduction
The simple linear regression model is y = 1 + 2 x + e .
The usual assumptions we make are
1. E (e) = 0
2. var(e) = 2
3. cov(ei , e j ) = 0
4. The variable x is not random, and it must take at least two different values.
5. (optional) e ~ N (0, 2 )
Slide 13.1
Undergraduate Econometrics, 2nd Edition-Chapter 13
Slide 13.2
Undergraduate Econometrics, 2nd Edition-Chapter 13
A13.1
yt = 1 + 2 xt + et .
A13.2
A13.3
x, is zero.
A13.4
A13.5
constant 2 .
A13.6
Slide 13.3
Undergraduate Econometrics, 2nd Edition-Chapter 13
Slide 13.4
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.2.1
The finite sample properties of the least squares estimator when x is random can be
summarized as follows:
13.2.2
What happens to the probability distribution of the least squares estimators if we have a
very large sample, or when the sample size T?
First, the least squares estimators are unbiased.
Second, the variances of the least squares estimators, given in equation 4.2.10,
converge to zero as T. As the sample size gets increasingly large the probability
distributions of the least squares estimators collapse about the true parameters.
Estimators with this property are called consistent estimators, and consistency is a nice
large sample property of the least squares estimators.
Slide 13.6
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.2.3
For the purposes of a large sample analysis of the least squares estimator we can
somewhat weaken the assumption A13.3. Let us replace it by
A13.3*
E (e ) = 0 and cov ( x, e ) = 0
Slide 13.7
Undergraduate Econometrics, 2nd Edition-Chapter 13
Slide 13.8
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.2.4
(13.2.1)
We have asterisked (*) the ability variable because it is difficult, if not impossible, to
observe.
Suppose that we attempt to measure ability using xt = a standardized test score. It is
sometimes called a proxy variable.
Slide 13.9
Undergraduate Econometrics, 2nd Edition-Chapter 13
xt = xt* + ut
(13.2.2)
= 1 + 2 ( xt ut ) + et
= 1 + 2 xt + (et 2ut )
(13.2.3)
= 1 + 2 xt + vt
Slide 13.10
Undergraduate Econometrics, 2nd Edition-Chapter 13
2
2 t
) =
2
2
u
(13.2.5)
( xt x ) .
2
Slide 13.11
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.2.5
Slide 13.12
Undergraduate Econometrics, 2nd Edition-Chapter 13
E x E ( x ) y E ( y ) = 2 E x E ( x ) + E x E ( x ) e ,
2
or
cov ( x, y ) = 2 var ( x ) + cov ( x, e )
Solve for 2
2 =
cov ( x, y ) cov ( x, e )
var ( x )
var ( x )
(13.2.6)
cov ( x, y )
.
var ( x )
(13.2.7)
Slide 13.13
Undergraduate Econometrics, 2nd Edition-Chapter 13
b2 =
x, y )
( xt x )( yt y ) ( xt x )( yt y ) / (T 1) cov(
=
=
2
2
x)
var(
( xt x )
( xt x ) / (T 1)
(13.2.8)
If cov ( x, e ) = 0 then
b2 =
x, y )
cov(
cov( x, y )
= 2
x)
var(
var( x)
2 =
cov ( x, y ) cov ( x, e )
var ( x )
var ( x )
Slide 13.14
b2
cov ( x, y )
cov ( x, e )
2
= 2 +
var ( x )
var ( x )
Slide 13.15
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.3.1
Variable
The k'th moment of a random variable is the expected value of the random variable
raised to the k'th power.
E (Y k ) = k = k'th moment of Y
(13.3.1)
Slide 13.16
Undergraduate Econometrics, 2nd Edition-Chapter 13
The k'th population moment in equation 13.3.1 can be estimated consistently using the
sample (of size T) analog
E (Y k ) = k = k'th sample moment of Y
(13.3.2)
=y
t =1
k
t
(13.3.3)
Slide 13.17
Undergraduate Econometrics, 2nd Edition-Chapter 13
In order to estimate the two population parameters and 2 we must equate two
population moments to two sample moments.
The first two population and sample moments of Y are:
T
E (Y ) = 1 = , = yt T
t =1
E (Y 2 ) = 2 ,
2 = yt2 T
(13.3.4)
t =1
= yt T = y
(13.3.5)
t =1
Slide 13.18
Undergraduate Econometrics, 2nd Edition-Chapter 13
Then, use equation 13.3.3, replacing the second population moment by its sample
value, and replacing the mean by equation (13.3.5)
" 2 = E (Y 2 ) 2
T
y
t =1
2
t
y2 =
y
t =1
2
t
Ty 2
(13.3.6)
( yt y )
In general, method of moments estimators are consistent in large samples, but there is
no guarantee that they are "best" in any sense.
Slide 13.19
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.3.2
(13.3.7)
(13.3.8)
1
( yt b1 b2 xt ) = 0
T
1
xt ( yt b1 b2 xt ) = 0
(13.3.9)
These two equations are identical to the least squares "normal" equations from Chapter
3, equation 3.3.6, and their solution yields the least squares estimators
b2 =
( xt x )( yt y )
2
x
x
( t )
b1 = y b2 x
Slide 13.21
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.3.3
Regression Model
Problems for least squares arise when x is random and correlated with the random
disturbance e, so that E ( xt et ) 0 .
Suppose, however, that there is another variable, zt, called an instrumental variable,
which satisfies the moment condition
E ( zt et ) = 0 E zt ( yt 1 2 xt ) = 0
(13.3.10)
Slide 13.22
Undergraduate Econometrics, 2nd Edition-Chapter 13
Then we can use the two equations 13.3.7 and 13.3.10 to obtain estimates of 1 and 2 .
The sample moment conditions are:
1
yt 1 2 xt = 0
T
1
zt yt 1 2 xt = 0
(13.3.11)
Solving these equations leads us to method of moments estimators, which are usually
called the instrumental variable estimators,
z
y T zt yt ( zt z )( yt y )
2 = t t
=
z
x
T
z
x
t t t t ( zt z )( xt x )
(13.3.12)
1 = y 2 x
Slide 13.23
Undergraduate Econometrics, 2nd Edition-Chapter 13
z
z
x
x
T
z
x
1
cov
,
(
)
( t )( t ) ( )
(13.3.13)
The sample covariance converges to the true covariance in large samples, so we can
say
cov ( z , y )
2
cov ( z , x )
(13.3.14)
For an instrumental variable to be valid, it must be uncorrelated with the error term e
but correlated with the explanatory variable x.
Slide 13.24
Undergraduate Econometrics, 2nd Edition-Chapter 13
2 =
cov ( z , y ) cov ( z , e )
cov ( z , x ) cov ( z , x )
(13.3.15)
If we can assume that that cov ( z , e ) = 0 , then the instrumental variables estimator in
equation (13.3.14) converges in large samples to 2,
cov ( z , y )
2
= 2 .
cov ( z , x )
(13.3.16)
In large samples the instrumental variable estimator has the approximate normal
distribution,
2
~ N ,
2
2 ( x x )2 r 2
t
zx
rzx2 is the sample correlation between the instrument z and the random regressor x.
We want an instrument that is highly correlated with z to improve the efficiency of the
instrumental variable estimator.
The error variance is estimated using the estimator
2IV =
yt 1 2 xt
T 2
(13.3.18)
Slide 13.26
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.3.4
Usually we have more instrumental variables at our disposal than are necessary.
For example, let w be a variable that is correlated with x but uncorrelated with e
E (et ) = E ( yt 1 2 xt ) = 0 yt 1 2 xt = m1 = 0
E ( zt et ) = E zt ( yt 1 2 xt ) = 0 zt yt 1 2 xt = m2 = 0
(13.3.19a)
(13.3.19b)
E ( wt et ) = E wt ( yt 1 2 xt ) = 0 wt yt 1 2 xt = m3 = 0 (13.3.19c)
Using the least squares principle, choose values for 1 and 2 that minimize the sum of
squares m12 + m22 + m32 .
It is best to use weighted least squares, with the weights being the reciprocals of the
variances of the moments, if the moments are uncorrelated.
Slide 13.27
Undergraduate Econometrics, 2nd Edition-Chapter 13
S 1 , 2
m12
m22
m32
=
+
+
var ( m1 ) var ( m2 ) var ( m3 )
(13.3.20)
The values of 1 and 2 that minimize this weighted sum of squares can be obtained
using a 2-step process.
(1) Regress x on a constant term, z and w, and obtain the predicted values x
(2) Use x as an instrumental variable for x.
Slide 13.28
Undergraduate Econometrics, 2nd Edition-Chapter 13
( y x ) = 0
x ( y x ) = 0
t
2 t
2 t
( x x )( y y ) = ( x x )( y y )
( x x )( x x ) ( x x )( x x )
2 =
(13.3.21)
1 = y 2 x
Slide 13.29
Undergraduate Econometrics, 2nd Edition-Chapter 13
These instrumental variables estimators, derived using the method of moments, are
also called two-stage least squares estimators, because they can be obtained using
two least squares regressions.
Stage 1 is the regression of x on a constant term, z and w, to obtain the predicted
values x
Stage 2 is ordinary least squares estimation of the simple linear regression
yt = 1 + 2 xt + errort
(13.3.22)
The approximate, large sample, variance of 2 is given by the usual formula for the
variance of the least squares estimator of equation (13.3.22),
( )
var 2
2
=
2
( xt x )
Slide 13.30
The estimator of the error variance must be based on the residuals from the original
model, yt = 1 + 2 xt + et ,
2
IV
yt 1 2 xt
T 2
( )
2 =
var
2IV
( xt x )
This estimated variance can be used as a basis for t-tests of significance and interval
estimation of parameters.
Slide 13.31
Undergraduate Econometrics, 2nd Edition-Chapter 13
13.4 Testing for Correlation Between Explanatory Variables and the Error Term
Slide 13.32
Undergraduate Econometrics, 2nd Edition-Chapter 13
There are several forms of the test, usually called the "Hausman Test," for this null and
alternative hypothesis
In the regression yt = 1 + 2 xt + et we wish to know whether x is correlated with e.
Let zt1 and zt 2 be instrumental variables for x.
At minimum you need one instrument for each variable you think might be correlated
with the error term. Then carry out the following steps:
Estimate the model xt = a0 + a1 zt1 + a2 zt 2 + vt by least squares, and obtain the
residuals vt = xt a0 + a1 zt1 + a2 zt 2 . If there are more than one explanatory variables
that are questionable, repeat this estimation for each one, using all available
instrumental variables in each regression.
Include the residuals computed in step 1 as an explanatory variable in the
regression, yt = 1 + 2 xt + vt + et . Estimate this "artificial regression" by least
squares, and employ the usual t-test for the hypothesis of significance
Slide 13.33
Undergraduate Econometrics, 2nd Edition-Chapter 13
If more than one variable is suspect, the test will be a F-test of joint significance of the
coefficients on the included residuals.
Slide 13.34
Undergraduate Econometrics, 2nd Edition-Chapter 13