Class Notes for Econometrics Expectation of Ordinary Least Squares under the Gauss Markov assumptions

Jean Eid
Assume that the following hold MLR1 Linearity in the parameter MLR2 Independence of the error term u and random sampling MLR3 No perfect Collinearity MLR3 Zero conditional mean

Expectation of the OLS Estimates

You need to go and work through the lecture notes for the derivation of OLS. I will be using some algebraic results from these notes in here but will not derive To simplify matters, let us suppose that we are estimating the following model yi = o + 1 xi + ui (1)

Also assume that MLR2 to MLR4 above hold. From the previous lecture notes we know what the OLS estimates of 0 and 1 are. 0 = y 1 x
N

yi (xi x) 1 =
i=1 N

(xi x)2
i=1

One property that we would like the estimator to have is to be unbiassed. This means that we expect the estimator to give us the correct value of the parameters. To have this property we need to check

 the expectation of 0 and 1 E 0 |x = E y 1 x|x N 1 |x = E E yi (xi x)

i=1 N

(2) x (3)

(xi x)2
i=1

yi (xi x) If we look at the denominator in (3), we have

i=1 N

. This quantity is only a function of the (xi x)2

i=1

x1 , x2 , x3 , , xN . However, since we are conditioning on x = [x1 , x2 , x3 , , xN ], this quantity is nonrandom. This means that we can take this quantity out of the E [] and we only need to calculate the conditional expectation of the numerator. Although the numerator is a function of x1 , x2 , x3 , , xN , it is also a function of y1 , y2 , y3 , , yN and therefore a random variable. We also know that the expectation operator is linear i.e. the expectation of the is the sum of the expectation. Using this and (3), we get E 1 |x = 1
N N

(xi x) E [yi |x]

2 i=1

(xi x)
i=1

Now concentrating on the E [yi |x] E [yi |x] = E [0 + 1 xi + ui |x] = 0 + 1 xi + E [ui |x] But MLR4; The zero conditional mean is that E [ui |x] = 0 Using this fact in (4), we get E [yi |x] = 1
N N

(4)

(xi x) (0 + 1 xi )
2 i=1

(xi x)
i=1 N

(xi x) = 0 i=1 N (xi x)2

i=1 i=1

(xi x) xi + 1 i=1 N (xi x)2 (5)

But
i=1

(xi x) = 0 and
N N

(xi x)2 =
i=1 i=1 N

x2 + x2 2xi x i x2 + N x2 2N x2 i
i=1 N

=
i=1 N

x2 N x2 i x2 x 2 i
i=1

= Using (6) in (4) we get

(6)

E 1 |x = 1

(7)

Notice that (??) says that the conditional expectation of 1 is equal to 1 and so it is independent of x1 , x2 , x3 , , xN . Therefore the unconditional expectation of 1 is also 1 . So nally we have E 1 = 1 Now using (7) in (2) we get E 0 |x = E [y|x] xE 1 |x = E [y|x] 1 x So we need to solve for E [y|x] E [y|x] = E 1 N
N

(8)

yi |x
i=1 N N N

1 = E N

0 + 1
i=1 N i=1

xi +
i=1

ui x

1 = 0 + 1 x + N = 0 + 1 x Using (9) in (8) we get

E [ui |x]
i=1

(9)

E 0 |x = 0

(10)

Similarly note that the conditional expectation of 0 does not depend on x, therefore the conditional 0 is 0 expectation of 3