Conditional Expectation

Linear Regression and the CEF

Modelo Linear
Modelo de Regresso e Esperana Condicional

Cristine Campos de Xavier Pinto

CEDEPLAR/UFMG

Maio/2010

Cristine Campos de Xavier Pinto Modelo Linear

Institute

Conditional Expectation

Linear Regression and the CEF

Our Problem
We have a vector of random variables (Y , X ) that follows a unknown population distribution, F (y , x, ). Statistical Inference: Use our sample (observed values of (Y , X )) to learn about . Econometrics: We are interested in the conditional distribution of a random variables given a set of other random variables. We can write the joint distribution as f (y , x, ) = f ( y j x, 1 ) f (x, 2 )

Cristine Campos de Xavier Pinto Modelo Linear

Institute

Conditional Expectation

Linear Regression and the CEF

Our Problem

In regression, we are interested in 1 . If there is no relationship between 1 and 2 , we can ignore f (x, 2 ). In general, we are not interested in the whole parameter vector 1 , but just in the rst moment and second moment. Regression Analysis: We do inference about E [ Y j X ]

Cristine Campos de Xavier Pinto Modelo Linear

Institute

Conditional Expectation

Linear Regression and the CEF

Conditional Expectation
Goal: Estimate the conditional expectation of Y conditional on X : E [ Y j X ] In general, we are interested in conditional expectations that allow us to infer causality from one explanatory variable (X1 ) to the response variable (Y ) : Structural Conditional Expectation.

Problems: We have observational data. The conditional expectation of interest depends on variables that we cannot observe. Some of the controls, we cannot observe. In addition, we can have measurement error or simultaneity problems. Solution: Under some identification assumptions, we can sometimes recover the structural conditional expectation of interest.
Cristine Campos de Xavier Pinto Modelo Linear Institute

Conditional Expectation

Linear Regression and the CEF

Why conditional expectation?

Theorem (CEF Decomposition): Any random variable can be decomposed into two parts: Yi = E [ Yi j Xi ] + i where

(i ) i is mean independent of Xi , E [ i j Xi ] = 0, and therefore (ii ) i is uncorrelated to any function of Xi , E [i h (Xi )] = 0

Proof. We need to use the law of iteration expectation.
Cristine Campos de Xavier Pinto Modelo Linear Institute

Conditional Expectation

Linear Regression and the CEF

Why conditional expectation?

CEF is the best prediction of Yi given Xi . As a consequence of the CEF decomposition, the CEF solves the minimum mean squared error prediction problem. Theorem Let m (Xi ) be any function of Xi . h E [ Yi j Xi ] = arg min E (Yi
m (X i )

m (Xi ))2

Proof. Exercise
Cristine Campos de Xavier Pinto Modelo Linear Institute

Conditional Expectation

Linear Regression and the CEF

Linear Regression and the CEF

CEF provides a natural summary of the empirical relationship between Y and X . There is a natural link between linear regression and CEF. We will summarize this relationship in 3 theorems.

Cristine Campos de Xavier Pinto Modelo Linear

Institute

Conditional Expectation

Linear Regression and the CEF

Theorem Suppose the CEF is linear. Then the CEF is the population regression function. Proof. Suppose that E [ Yi j Xi ] = Xi , is a vector kx1. By the CEF decomposition property 0 = E [Xi (Yi E [ Yi j Xi ])] = E [Xi (Yi
1

Xi )]

Assuming that E [Xi Xi] is nonsingular, = E [Xi Xi]

Cristine Campos de Xavier Pinto Modelo Linear

E [Xi Yi ] =
Institute

where is the population regression coe cient vector

Conditional Expectation

Linear Regression and the CEF

Linear Regression and the CEF

Question: What makes de CEF linear?

1 2

Classic Situation: (Yi , Xi ) has a multivariate normal distribution. Regression models that are saturated (models that have a separate parameter for every combination of values that the set of regressors can take on) The population regression provide the best linear approximation for the CEF, and the CEF is the best unrestricted predictor of Y given X .

Cristine Campos de Xavier Pinto Modelo Linear

Institute

Conditional Expectation

Linear Regression and the CEF

Theorem The function Xi is the best linear predictor of Y given X , which means that it is the function that minimizes the mean square error. Proof. We need to nd the solution of the following problem i h min E (Yi Xib )2
b

The rst order condition is

E [Xi (Yi

Xib )] = 0

Assuming that E [Xi Xi] is nonsingular, b = E [Xi Xi]

Cristine Campos de Xavier Pinto Modelo Linear

E [Xi Yi ] =
Institute

Conditional Expectation

Linear Regression and the CEF

Theorem The function Xi provides the minimum mean square error linear approximation to E [ Yi j Xi ], i h = arg min E (E [ Yi j Xi ] Xib )2
b

Cristine Campos de Xavier Pinto Modelo Linear

Institute

Conditional Expectation

Linear Regression and the CEF

Proof: We know that solves the following problem h i min E (Yi Xib )2
b

Note that i h h E (Yi Xib )2 = E (Yi h = E (Yi

In this case, h arg min E (Yi

b
Cristine Campos de Xavier Pinto Modelo Linear

+2 E [(Yi E [ Yi j Xi ]) (E [ Yi j Xi ] | {z =0 i h +E (E [ Yi j Xi ] Xib )2
Xib )2 i

E [ Yi j Xi ] + E [ Yi j Xi ] i E [ Yi j Xi ])2

Xib )2

Xib )] }

i h Xib )2 = arg min E (E [ Yi j Xi ]

b

Institute

Conditional Expectation

Linear Regression and the CEF

Summary

CEF is the best unrestricted predictor of Y given X . Regression provides the best linear predictor of Y given X . Regression provides the best linear approximation to CEF. If we are interested in estimating E [ Yi j Xi ], we can use a linear regression to approximate it.

Cristine Campos de Xavier Pinto Modelo Linear

Institute

Conditional Expectation

Linear Regression and the CEF

References

Amemya: 1 Goldberger: 5 Hayashi: 1 and 2 Rudd: 1 and 6 Wooldridge: 2

Cristine Campos de Xavier Pinto Modelo Linear

Institute