Econometrics

Hypothesis Testing (Inference) in Linear Regression Models Yuyi LI

University of Manchester

2012

Yuyi LI (University of Manchester)

Inference in Linear Regression Models

2012

1 / 27

Outline
Finite-sample inference in linear models: the normality assumption Sampling distributions of the OLS estimators Hypothesis tests: a single restriction The t test Hypothesis tests: multiple linear restrictions The F test Reading: Wooldridge Chapter 4, Appendix C6 Mathematics and Statistics: standard normal, t and F distributions, statistical hypotheses, test statistics, rejection rules, signicance level, critical values, p-value, degrees of freedom. . .
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 2 / 27

Finite-Sample Distributions of OLS Estimators

Normality in Finite Samples

Assumption MLR.6: Normality

Recall the MLR model y = 0 + 1 x1 + 2 x2 + . . . + k xk + u Can test hypotheses about j based on the sample distribution of its estimator j . Need an extra assumption on u in nite samples: Assumption MLR.6: (Normality) Error term u is independent of the regressors x1 , x2 , . . . , xk and is normally distributed: u N (0, 2 ). MLR.6 can be justied by the Central Limit Theorem as sample size tends to innity. MLR.1-6 are the Classical Linear Model (CLM) assumptions. Inference in Linear Regression Models Yuyi LI (University of Manchester) 2012 3 / 27

Finite-Sample Distributions of OLS Estimators

Normality in Finite Samples

Normality of OLS Estimators

Normality of u translates to OLS estimators. Theorem (Normality of OLS Estimators) Under CLM assumptions (MLR.1-MLR.6),

j N (j , Var (j )),

j = 0, 1, 2, . . . , k

where Var (j ) = 2 (X X)1 and (X X)1 denotes the (j + 1)-th jj jj leading diagonal element of (X X)1 .
Then, an infeasible distribution follows

j j Var (j )
Yuyi LI (University of Manchester)

N (0, 1)

Inference in Linear Regression Models

2012

4 / 27

Finite-Sample Distributions of OLS Estimators

Normality in Finite Samples

The t Distribution of OLS Estimators

Estimate Var (j ) : Var (j ) = 2 (X X)1 = [se (j )]2 jj where 2 = SSR /(n 1 k )
Theorem (t Distribution of OLS Estimators) Under the CLM assumptions (MLR.1-MLR.6),

j j tn1k , for j = 0, 1, . . . , k se (j )
where tdf is a t distribution with df degrees of freedom tdf approximates N (0, 1) when df is large (> 120) This can be used to test hypothesis on j
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 5 / 27

Finite-Sample Distributions of OLS Estimators

Hypothesis Testing (Inference)

Hypotheses
Consider MLR model y = 0 + 1 x1 + 2 x2 + . . . + k xk + u The null and alternative hypotheses (H0 and H1 ) Examples of hypotheses:
(i) H0 : 1 = 0 (a restriction on one parameter) x1 has no impact on y (ii) H0 : 1 2 = 0 (a linear restriction on parameters) x1 and x2 have equivalent effects on y (iii) H0 : 1 = 2 = k = 0 (multiple linear restrictions) x1 , x2 and xk have no joint explanatory power on y

What test to use?

t : (i) a restriction on one parameter or (ii) a linear restriction on parameters F : (iii) multiple linear restrictions (and (i) & (ii))
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 6 / 27

Finite-Sample Distributions of OLS Estimators

Hypothesis Testing (Inference)

Classical Testing Procedure

The Classical procedure involves steps to
Select signicance level (): probability of rejecting H0 when it is true. Common choices: 1%, 5%, 10% Choose H1 : may affect the rejection rule Calculate the test statistic Reject H0 if the calculated test statistic is in the rejection region; Otherwise, do not reject H0

Note, selection of is somewhat arbitrary Note, rejection decision is made by comparing test statistic and corresponding critical value Note, critical values depend on , H1 , and statistic distribution under H0
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 7 / 27

Finite-Sample Distributions of OLS Estimators

Hypothesis Testing (Inference)

Hypothesis Testing using the p-Value

Choice of (e.g. 5%) is somewhat arbitrary

An alternative: Given the test statistic, what is the smallest signicance level at which H0 would be rejected? This is called the p-value of the test. The p-value (p) is the probability of observing a test statistic (T ) as extreme as we did if H0 is true:
E.g. t test for H0 : j = 0 against H1 : j 0 = p = Pr(|T | > |t |), where t is the observed t test statistic

Small p-values provides evidence against H0

E.g. H0 : 1 = 0, H1 : 1 0, t t930 , observed value of t = 2.137642 and p = 0.0328. Then H0 is rejected at 5% level, but not at 1% level. Precisely, H0 is rejected at all signicance levels 3.28%
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 8 / 27

The t Test

A Single Parameter

t Test: A Restriction on One Parameter

The null hypothesis (j = 0, 1, 2, . . . , k ): H0 : j = c , The t statistic t= where c is a constant

j j j c = tn1k under H0 se (j ) se (j )

(1)

Example: if j = 0.8 and se (j ) = 0.2, the test statistic for H0 : j = 1 is t = (0.8 1)/0.2 = 1 Next, we introduce a special case when c = 0
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 9 / 27

The t Test

A Single Parameter

Signicant Regressor & Rejection Rule

If c = 0, then H0 : j = 0. H0 : once x1 , x2 , . . . , xj 1 , xj +1 , . . . , xk have been controlled for, xj has no effect on the expected value of y (i.e. xj is statistically insignicant). The t statistic is

j j j 0 j t= = = tn1k under H0 se (j ) se (j ) se (j )
These t statistics are routinely reported in EViews Alternatives and Rejection Rules:
Alternative Rejection Rule
Yuyi LI (University of Manchester)

H1 : j > 0 t > tcv

H1 : j < 0 t < tcv

H1 : j 0 |t | > tcv
2012 10 / 27

Inference in Linear Regression Models

The t Test

A Single Parameter

Example
Test H0 : 1 = 1 against H1 : 1 < 1 with = 0.01 y = 0 + 1 x1 + 2 x2 + u y = 0.2 + 0.8 x1 1.2 x2
(0.03) (0.2) (0.05)

where standard errors are in brackets, n = 353 t test statistic t =

1 1 tn12 = t350 under H0 se (1 ) 0.8 1 = = 1 0 .2

Critical value tcv = 2.326 (from t350 , 1-tailed H1 , = 0.01) Reject H0 if t < tcv : Fail to reject H0 , as t > tcv here
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 11 / 27

The t Test

A Single Linear Combination of Parameters

A Linear Restriction on Parameters

H0 involves several parameters, but only one restriction: e.g. H0 : 1 = 2 or H0 : 1 + 2 = 3 Example: Cobb-Douglas production function Yi = ALi 1 Ki 2 Ui where Y =production, A =technology, L =labour, K =capital and U=unobservables. Taking logs: log(Yi ) = log(A ) + 1 log(Li ) + 2 log(Ki ) + ui Constant returns to scale: H0 : 1 + 2 = 1. Note, t tests can be used to test H0 , but statistics are not easily computed due to the complexity of corresponding standard errors
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 12 / 27

The t Test

A Single Linear Combination of Parameters

Equivalence of Educations
Consider the population model log(wage ) = 0 + 1 x1 + 2 x2 + 3 x3 + u
x1 =the number of years in junior college x2 =the number of years in university x3 =the number of months in workforce

(2)

H0 : 1 = 2 . What does it mean? H1 : 1 < 2 . What does this imply? The hypotheses can be rewritten as H0 : 1 2 = 0, H1 : 1 2 < 0.
The following slides are based on Wooldridge section 4.4, which mainly details two methods of obtaining the standard error of a linear combination of paramter estimators. Wooldrige section 4.4 (page 140- )
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 13 / 27

The t Test

A Single Linear Combination of Parameters

A t Statistic and A Problem

A t statistic can be constructed to test H0 : t=

1 2 se (1 2 )

(3)

where estimates 1 and 2 are easily obtained. Problem: How to get se (1 2 )?

Estimate the variance of (1 2 ): Var (1 2 ) = Var (1 ) + Var (2 ) 2Cov (1 , 2 ) se (1 2 ) =
=

Var (1 ) + Var (2 ) 2Cov (1 , 2 )

[se (1 )]2 + [se (2 )]2 2s12 , where a hat indicates an unbiased estimator and s12 = Cov (1 , 2 ).
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 14 / 27

The t Test

A Single Linear Combination of Parameters

The Problem and Solutions

Note, se (1 ) and se (2 ) are computed by any software, but s12 is not = se (1 2 ) on the previous page is unknown = Statistic (3) is not computable = Testing H0 is not possible? Solutions: Two methods: Method 1: Compute s12 by estimating the covariance matrix of Method 2: Rewrite model (2) by introducing a new parameter These two methods are explained sequentially. . .
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 15 / 27

The t Test

A Single Linear Combination of Parameters

Method One (1/2)

Write model (2) in matrix form (Revise notes L3) y = X + u
where y = [y1 , y2 , , yn ] , y log(wage ), X = [x1 , x2 , , xn ] , xi = [1, xi1 , xi2 , xi3 ], i = 1, 2, . . . , n, = [0 , 1 , 2 , 3 ] and u = [u1 , u2 , , un ]

OLS estimator is = (X X)1 X y. Covariance matrix is Var (|X) = 2 (X X)1 , and an unbiased estimator is SSR (X X)1 Var (|X) = 2 (X X)1 = n1k What is the structure of Var (|X)?
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 16 / 27

The t Test

A Single Linear Combination of Parameters

Method One (2/2)

Var (|X) = [se (0 )]2 s01 s02 2 s10 [se (1 )] s12 s20 s21 [se (2 )]2
s30 s31 s32 s03 s13 s23 [se (3 )]2

If we compute Var (|X), it is straightforward to obtain se (1 ), se (2 ) and s12 = se (1 2 ) can be obtained = Statistic (3) can be computed = Testing H0 is possible! is 4 1 here. How to generalise to (k + 1) 1?
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 17 / 27

The t Test

A Single Linear Combination of Parameters

Method Two (1/2)

Dene = 1 2 = (i) H0 : = 0 is the same as H0 : 1 2 = 0 = (ii) 1 = + 2 = (iii) statistic (3) becomes t = /se () Note: Computing (iii) to test (i) is now the solution! How to compute (iii)? Rewrite model (2): y = 0 + 1 x1 + 2 x2 + 3 x3 + u = 0 + ( + 2 )x1 + 2 x2 + 3 x3 + u = 0 + x1 + 2 x1 + 2 x2 + 3 x3 + u = 0 + x1 + 2 (x1 + x2 ) + 3 x3 + u Note, models (2) and (4) are equivalent
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 18 / 27

(4)

The t Test

A Single Linear Combination of Parameters

Method Two (2/2)

Model (4) can be estimated by regressing y on an intercept, x1 , (x1 + x2 ) and x3 = This yields and se () directly = Statistic t = /se () in (iii) can be computed = Testing H0 : = 0 in (i) is possible! Either method enables a t test on hypothesis involving a linear restriction on several parameters Note, t test is used to test a single linear restriction with one or more parameters Note, if there are multiple linear restrictions, F test can be used
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 19 / 27

The F Test

Multiple Linear Restrictions

Multiple Linear Restrictions

Consider a model y = 0 + 1 x1 + 2 x2 + 3 x3 + 4 x4 + u Example: q linear restrictions hypotheses
H0 : 1 = 3 = 0 H0 : 1 = 0, 2 = 3 H0 : 1 = 2 = 3 = 4 = 0 exclusion restrictions two restrictions reg. overall signicance

(5)

What is q under each of the above H0 ? Alternative hypotheses: H1 : H0 does not hold. Note, H0 is violated as long as one restriction fails To construct the F test, we need to identify the restricted model and unrestricted model. . .
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 20 / 27

The F Test

Multiple Linear Restrictions

Restricted and Unrestricted Models

Restricted model: (5) with q restrictions under H0 Example, restricted models are
H0 : 1 = 3 = 0 (5) = y = 0 + 2 x2 + 4 x4 + u H0 : 1 = 0, 2 = 3 (5) = y = 0 + 2 (x2 + x3 ) + 4 x4 + u H0 : 1 = 2 = 3 = 4 = 0 (5) = y = 0 + u

Unrestricted model: (5) without any restrictions Estimating restricted model yields SSRr (Sum of Squared Residuals in the restricted model) Estimating unrestricted model yields SSRur (Sum of Squared Residuals in the unrestricted model) Always, SSRur SSRr .
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 21 / 27

The F Test

Multiple Linear Restrictions

The F Statistic and Rejection Rule

The F statistic F=

(SSRr SSRur )/q Fq,n1k under H0 SSRur /(n 1 k )

where q = number of restrictions under H0 (numerator df ), n 1 k = unrestricted model df (denominator df ), Fq,n1k = an F distribution with q and (n 1 k ) df df = degrees of freedom

F 0. Why? Reject H0 if F > Fcv (critical value from Fq,n1k ). Fail to reject H0 otherwise.
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 22 / 27

The F Test

Multiple Linear Restrictions

An Example for F Test 1/2

Consider the wage equation model lw = 0 + 1 ed + 2 ex + 3 te + 4 hr + u (6)

where lw = log (wage ) (monthly), ed = years in education, ex = years of work experience, te = years in the current employment, hr = average weekly working hours, and u denotes the random error term.

te and hr are irrelevant to wage (dropped in (6))? H0 : 3 = 4 = 0 against H1 : 3 0 or/and 4 0. OLS regression of (unrestricted) model (6): SSRur Restricted model (under H0 ; 2 restrictions, q = 2) lw = 0 + 1 ed + 2 ex + u: SSRr .
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 23 / 27

The F Test

Multiple Linear Restrictions

An Example for F Test 2/2

In our case, n = 935, k = 4, q = 2, SSRur = 139.28, SSRr = 143.98. The F statistic is
F=

(SSRr SSRur )/q (143.98 139.28)/2 = 15.70 SSRur /(n 1 k ) 139.28/(935 1 4)

Critical value Fcv = 4.61 (from F2,930 , = 0.01) Since F 15.70 > Fcv , H0 is rejected at 1% signicance level Statistically, years in the current employment (te) and average weekly working hours (hr) have some joint effect on wage.
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 24 / 27

The F Test

Multiple Linear Restrictions

The R-Squared Form of the F Statistic

The F statistic can also be computed based on the R 2 values in the unrestricted and restricted models
2 (Rur Rr2 )/q F= 2 (1 Rur )/(n 1 k ) 2 where Rur is the R 2 from the unrestricted model and Rr2 is from the restricted model. Note, this formula works only if unrestricted and restricted models have same dependent variables For example, R 2 form F statistic is invalid to test general linear hypothesis H0 : 0 = 0, 1 2 = 1
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 25 / 27

The F Test

Multiple Linear Restrictions

Overall Signicance of the Model

F test for the joint signicance of all the regressors (excluding the constant) Consider (unrestricted) model: y = 0 + 1 x1 + 2 x2 + . . . + k xk + u. H0 : 1 = 2 = . . . = k = 0 against H1 : any j 0, j = 1, 2, . . . , k Restricted model: y = 0 + e; Note, Rr2 = 0. why? R 2 form of F statistic 2 2 (Rur Rr2 )/q Rur /q F= = 2 2 (1 Rur )/(n 1 k ) (1 Rur )/(n 1 k ) This statistic is routinely reported in EViews and most statistical softwares. Can you nd it?
Yuyi LI (University of Manchester) Inference in Linear Regression Models 2012 26 / 27

Appendix

Deriving R-Squared Form of the F Statistic

R-Squared Form of the F Statistic. By denition, R2 = SSE SSR =1 = SSR = SST (1 R 2 ) SST SST

2 This means, SSRur = SST (1 Rur ), SSRr = SST (1 Rr2 ) in the unrestricted and restricted models. Then F statistic becomes 2 [SST (1 Rr2 ) SST (1 Rur )]/q (SSRr SSRur )/q = 2 SSRur /(n 1 k ) [SST (1 Rur )]/(n 1 k ) 2 2 [(1 Rr2 ) (1 Rur )]/q (Rur Rr2 )/q = = 2 2 (1 Rur )/(n 1 k ) (1 Rur )/(n 1 k )

F =

Yuyi LI (University of Manchester)

Inference in Linear Regression Models

2012

27 / 27