Multiple Regression Analysis

y = b0 + b1x1 + b2x2 + . . . bkxk + u

6. Heteroskedasticity

Economics 20 - Prof. Anderson

What is Heteroskedasticity
2

Recall the assumption of homoskedasticity implied that conditional on the explanatory variables, the variance of the unobserved error, u, was constant If this is not true, that is if the variance of u is different for different values of the xs, then the errors are heteroskedastic Example: estimating returns to education and ability is unobservable, and think the variance in ability differs by educational attainment
Economics 20 - Prof. Anderson

Example of Heteroskedasticity
f(y|x)

.
x1
3

.
x3
Economics 20 - Prof. Anderson

E(y|x) = b0 + b1x

x2

Why Worry About Heteroskedasticity?

4

OLS is still unbiased and consistent, even if we do not assume homoskedasticity The standard errors of the estimates are biased if we have heteroskedasticity If the standard errors are biased, we can not use the usual t statistics or F statistics or LM statistics for drawing inferences

Economics 20 - Prof. Anderson

Variance with Heteroskedasticity

5

xi x ui For the simple case, b1 b1 , so 2 xi x

Var b 1

2 x x i i 2

SST
2

2 x

, where SSTx xi x
2 i 2

A valid estimator for this when is

2 x x u i i

SST

2 x

i are are the OLS residuals , where u

Economics 20 - Prof. Anderson

Variance with Heteroskedasticity

6

For the general multiple regression model, a valid with heteroskedasticity is estimator of Var b

b Var j

2 r u ij i

SST

2 j

th , where rij is the i residual from

regressing x j on all other independent variables, and SST j is the sum of squared residuals from this regression
Economics 20 - Prof. Anderson

Robust Standard Errors

7

Now that we have a consistent estimate of the variance, the square root can be used as a standard error for inference Typically call these robust standard errors Sometimes the estimated variance is corrected for degrees of freedom by multiplying by n/(n k 1) As n its all the same, though

Economics 20 - Prof. Anderson

Robust Standard Errors (cont)

8

Important to remember that these robust standard errors only have asymptotic justification with small sample sizes t statistics formed with robust standard errors will not have a distribution close to the t, and inferences will not be correct In Stata, robust standard errors are easily obtained using the robust option of reg

Economics 20 - Prof. Anderson

A Robust LM Statistic
9

Run OLS on the restricted model and save the residuals Regress each of the excluded variables on all of the included variables (q different regressions) and save each set of residuals 1, 2, , q Regress a variable defined to be = 1 on 1 , 2 , , q , with no intercept The LM statistic is n SSR1, where SSR1 is the sum of squared residuals from this final regression
Economics 20 - Prof. Anderson

10

Example 8.3: Heteroskedasticity-Robust LM Statistic

use "E:\Bahan Ajar\2012\Econometrics\Woolridge data\crime1.dta", clear gen avgsensq=avgsen*avgsen reg narr86 pcnv avgsen avgsensq ptime86 qemp86 inc86 black hispan, robust di _b[avgsen]/(2*_b[avgsensq]) reg narr86 pcnv ptime86 qemp86 inc86 black hispan predict ubar1, resid quiet reg avgsen pcnv ptime86 qemp86 inc86 black hispan predict r1, r quiet reg avgsensq pcnv ptime86 qemp86 inc86 black hispan predict r2, r quiet gen ur1 = ubar1*r1 quiet gen ur2 = ubar1*r2 gen iota = 1 reg iota ur1 ur2, noconstant quiet scalar hetlm = e(N)-e(rss) quiet scalar pval = chi2tail(2,hetlm) display "Robust LM statistic : " %6.3f hetlm " " "Under H0, distrib Chi2(2), p-value: " %5.3f pval

Economics 20 - Prof. Anderson

Testing for Heteroskedasticity

11

Essentially want to test H0: Var(u|x1, x2,, xk) = 2, which is equivalent to H0: E(u2|x1, x2,, xk) = E(u2) = 2 If assume the relationship between u2 and xj will be linear, can test as a linear restriction So, for u2 = d0 + d1x1 ++ dk xk + v) this means testing H0: d1 = d2 = = dk = 0

Economics 20 - Prof. Anderson

The Breusch-Pagan Test

12

Dont observe the error, but can estimate it with the residuals from the OLS regression After regressing the residuals squared on all of the xs, can use the R2 to form an F or LM test The F statistic is just the reported F statistic for overall significance of the regression, F = [R2/k]/[(1 R2)/(n k 1)], which is distributed Fk, n k - 1 The LM statistic is LM = nR2, which is distributed c2k
Economics 20 - Prof. Anderson

The White Test

13

The Breusch-Pagan test will detect any linear forms of heteroskedasticity The White test allows for nonlinearities by using squares and crossproducts of all the xs Still just using an F or LM to test whether all the xj, xj2, and xjxh are jointly significant This can get to be unwieldy pretty quickly
Economics 20 - Prof. Anderson

Alternate form of the White test

14

Consider that the fitted values from OLS, , are a function of all the xs Thus, 2 will be a function of the squares and crossproducts and and 2 can proxy for all of the xj, xj2, and xjxh, so Regress the residuals squared on and 2 and use the R2 to form an F or LM statistic Note only testing for 2 restrictions now

Economics 20 - Prof. Anderson

Weighted Least Squares

15

While its always possible to estimate robust standard errors for OLS estimates, if we know something about the specific form of the heteroskedasticity, we can obtain more efficient estimates than OLS The basic idea is going to be to transform the model into one that has homoskedastic errors called weighted least squares

Economics 20 - Prof. Anderson

Case of form being known up to a multiplicative constant

16

Suppose the heteroskedasticity can be modeled as Var(u|x) = 2h(x), where the trick is to figure out what h(x) hi looks like E(ui/hi|x) = 0, because hi is only a function of x, and Var(ui/hi|x) = 2, because we know Var(u|x) = 2hi So, if we divided our whole equation by hi we would have a model where the error is homoskedastic
Economics 20 - Prof. Anderson

Generalized Least Squares

17

Estimating the transformed equation by OLS is an example of generalized least squares (GLS) GLS will be BLUE in this case GLS is a weighted least squares (WLS) procedure where each squared residual is weighted by the inverse of Var(ui|xi)

Economics 20 - Prof. Anderson

Weighted Least Squares

18

While it is intuitive to see why performing OLS on a transformed equation is appropriate, it can be tedious to do the transformation Weighted least squares is a way of getting the same thing, without the transformation Idea is to minimize the weighted sum of squares (weighted by 1/hi)

Economics 20 - Prof. Anderson

More on WLS
19

WLS is great if we know what Var(ui|xi) looks like In most cases, wont know form of heteroskedasticity Example where do is if data is aggregated, but model is individual level Want to weight each aggregate observation by the inverse of the number of individuals

Economics 20 - Prof. Anderson

Feasible GLS
20

More typical is the case where you dont know the form of the heteroskedasticity In this case, you need to estimate h(xi) Typically, we start with the assumption of a fairly flexible model, such as Var(u|x) = 2exp(d0 + d1x1 + + dkxk) Since we dont know the d, must estimate

Economics 20 - Prof. Anderson

Feasible GLS (continued)

21

Our assumption implies that u2 = 2exp(d0 + d1x1 + + dkxk)v Where E(v|x) = 1, then if E(v) = 1 ln(u2) = a0 + d1x1 + + dkxk + e Where E(e) = 1 and e is independent of x Now, we know that is an estimate of u, so we can estimate this by OLS

Economics 20 - Prof. Anderson

Feasible GLS (continued)

22

Now, an estimate of h is obtained as = exp(), and the inverse of this is our weight So, what did we do? Run the original OLS model, save the residuals, , square them and take the log reg y x1 .. Xk predict uhat, resid gen lnuhatsq=log(uhat^2) Regress ln(2) on all of the independent variables and get the fitted values, reg lnuhatsq x1 xk predict ghat gen hhat=exp(ghat) Do WLS using 1/exp() as the weight
Economics 20 - Prof. Anderson

WLS Wrapup
23

When doing F tests with WLS, form the weights from the unrestricted model and use those weights to do WLS on the restricted model as well as the unrestricted model Remember we are using WLS just for efficiency OLS is still unbiased & consistent Estimates will still be different due to sampling error, but if they are very different then its likely that some other Gauss-Markov assumption is false
Economics 20 - Prof. Anderson