Test PDF

Outline
Test on single parameters

Test on more parameters
Test: matrix algebra approach
Test on regression parameters

Prof. Rizzi Laura
October 28, 2013
Prof. Rizzi Laura
Outline
Overview on:
I
Prof. Rizzi Laura
Outline

(See 5.6 Stock and Watson)
b
We assessed that jvar (bj ) has approximately standard normal distribution (due to Central Limit
j
Theorem), then the hypothesis on single parameters j of the regression models may be verified
using the test statistic z or t (the latter is used when 2 is unknown).
The confidence interval for parameter j may be derived following the usual approach, then is
defined as {bj 1.96 s.e.(bj )}. This holds for each of the parameters 1 , 2 , ..., k .
Example test-STR
The estimated models were:
i =
Test
i = 696
Test
(8.7)
689.9
2.28 STRi
(10.4)
(0.52)
1.1 STRi
(0.43)
Prof. Rizzi Laura
0.65 PctELi
(0.031)
Outline

If we consider the variable EXPN, which measures the student/expenditure, to include into the
model we obtain:
i = 1 + 2 STRi + 3 EXPNi + 4 PctELi
Test
The hypothesis that district expenditure per student does not affect the performances of students
may be expressed as follows:
H0 : 2 = 0, 3 = 0
H1 : 2 6= 0 3 6= 0
This hypothesis involves more parameters, moreover it is not a single hypothesis on more
parameters but a multiple hypothesis involving more single hypothesis. It involves more
restrictions on more that one parameters.
We could think to verifiy this multiple hypothesis taking into consideration the test statistic t
relative to each single hypothesis involved. Rejecting thus the null if one of the test t relative to
single parameters has sample value belonging to the rejection tails.
Prof. Rizzi Laura
Outline

This approach is not correct because the final multiple test statistic derived would not have the
correct significance level.
Proof:
Consider the null hypothesis H0 : 2 = 0 and 3 = 0 and derive the significance level based on the
consideration of the single test statistics t.
Suppose that b2 and b3 have independent sampling distributions, then the relative test statistics t
are:
t2 =
b2 0
s.e.(b2 )
t3 =
b3 0
s.e.(b3 )
The decision rule could be:

reject H0 : 2 = 3 = 0 if |t2 | > t and/or |t3 | > t
2
Which is the probability to reject the null, even a true null? Which is thus the first type error
probability? (It should be equal to anyway)
Prof. Rizzi Laura
Outline

This probability is the following:
PrH0
[|t2 | > 1.96, |t3 | > 1.96] + PrH0 [|t2 | > 1.96, |t3 | 1.96] +
PrH0 [|t2 | 1.96, |t3 | > 1.96]
PrH0 [|t2 | > 1.96] PrH0 [|t3 | > 1.96] +
PrH0 [|t2 | > 1.96] PrH0 [|t3 | 1.96] +
PrH0 [|t2 | 1.96] PrH0 [|t3 | > 1.96]
With independent t2 and t3 we obtain:

= 0.05 0.05 + 0.05 0.95 + 0.95 0.05 = 0.0975 = 9.75
Which is not equal to = 0.05. The size depends on the correlation between t2 and t3 , then on
the correlation between b2 and b3 .
Prof. Rizzi Laura
Outline

The possible solutions of this problem are:
I to consider a critical value different from
2 (Bonferroni approach);
I to consider a different test statistic, involving at the same time more than one hypothesis on
different parameters (the test statistic F).
The test F derived in Stock and Watson (see 5.7)

The test statistic F verifies multiple hypothesis on more than two parameters. If the multiple null
hypothesis is for instance 1 = 1,0 and 2 = 2,0 in a model with two regressors (k = 3) the test
F is:
!
1 t12 + t22 2t1 ,t2 t1 t2
F =
2
1 t1 ,t2
Where t1 ,t2 is the estimated correlation between t1 and t2 . The test F takes hight sampling values
when t1 and/or t2 have hight values. The test F adjust for the correlation between t1 and t2 . The
expression of F statistic becomes more complicated when more than two parameters are involved
in the null hypothesis (in these situations it is better to consider the matrix algebra approach)
Prof. Rizzi Laura
Outline
F distribution in large samples

p
If t1 and t2 are independent t1 ,t2 0, then in large samples the test F expression becomes:
1
2
t12 + t22 2t1 ,t2 t1 t2

1 t1 ,t2
1 2
2
= (t1 + t2 )
2
The F test statistic in large samples is the average of the two test statistic t.
Stata software example
The Stata statistical software requires the command test after regression estimation to apply test
statistic F to verify every null hypothesis. We verify the multiple hypothesis for the significance of
the regression coefficients related to the covariates STR e EXPN.
Prof. Rizzi Laura
Outline
Stata software example
reg testscr str expn_stu pctel, r;

Regression with robust standard errors
Number of obs
F( 3,
416)
Prob > F
R-squared
Root MSE
=
=
=
=
=
420
147.20
0.0000
0.4366
14.353
-----------------------------------------------------------------------------|
Robust
testscr |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------str | -.2863992
.4820728
-0.59
0.553
-1.234001
.661203
expn_stu |
.0038679
.0015807
2.45
0.015
.0007607
.0069751
pctel | -.6560227
.0317844
-20.64
0.000
-.7185008
-.5935446
_cons |
649.5779
15.45834
42.02
0.000
619.1917
679.9641
-----------------------------------------------------------------------------NOTE
test str expn_stu;
( 1)
( 2)
The test command follows the regression
str = 0.0
expn_stu = 0.0
F(
2,
416) =
Prob > F =
There are q=2 restrictions being tested

5.43
0.0047
The 5% critical value for q=2 is 3.00

Stata computes the p-value for you
Prof. Rizzi Laura
Outline

We may verify different hypothesis on the parameters of a regression model:
I H0 : j = 0 against H1 : j 6= 0, this is a significance test on the parameter j ;
I H0 : j = j,0 against H1 : j 6= j,0 , in this case we verify if the parameter j is equal to
j,0 ;
I H0 : j + i = 1 against H1 : j + i 6= 1;
I H0 : j = i against H1 : j 6= i
These are single hypothesis, which involve one or more model parameters. There may be multiple
hypothesis which involve more single hypothesis on the parameters at the same time:

0
2
3 0

I H0 : ~2 =
. = .
.. ..
k
0
Prof. Rizzi Laura
Outline

In the last case we verify the significance of all regression coefficients.
~ as
In other cases we consider the significance of a subset of parameters (partitioning the vector )
follows:
~1
~=
~2
In this situation ~1 and ~2 are vectors of order (k1 1) and ((k k1 ) 1), respectively, and the
null hypothesis may involve the significance of the parameters included in one of the sub-vectors.
Prof. Rizzi Laura
Outline

All the hypothesis may be expressed in matrix algebra terms:
~
~ = ~r
R
Where:
~ is a matrix (q k) of constants;
I R
I ~r is a vector (q 1) of constants;
~ is the (k 1) vector of the model parameters;
I
I q are the rows of the matrix R and gives also the number of single hypothesis on the
parameters (restrictions) involved (it must be that q < k)
Consider again the hypothesis above in matrix algebra terms:
[1] - H0 : j = 0

~ =
0
0 1 0
R
~ is a (1 k) matrix.
Where ~r = 0, q = 1 then R
Prof. Rizzi Laura
Outline

[2]- H0 : j = j,0
~ =
R
Where ~r = j,0 , q = 1 then R
[3]- H0 : 2 + 3 = 1
~ =
R
Where ~r = 1, q = 1 then R
[4]- H0 : 2 = 3 o H0 : 2 3 = 0
~ =
R
Where ~r = 0 and q = 1.
Prof. Rizzi Laura
Outline

[5]- H0 : ~2 = 0
2
3
~2 = .
..
k
~ =
R
~0
~Ik1
Prof. Rizzi Laura
0
0

= .
..
0
0
0
1
0
0
1
0
.
.
.
0
0
.
.
.
0
0
.
.
.
0
..
.
.
.
.
0
0
.
.
.
1
Outline

The second part of matrix R is an identity matrix with order ((k 1) (k 1)), in general the
matrix R has order (k 1) k, moreover the vector ~r = 0 is a ((k 1) 1) vector with al zero
values and q = k 1.
~
~ = ~r or as R
~
~ ~r = 0.
All restriction, then all hypothesis, may be expressed as R
The general test may be specialized to deal with any specification.
~
~ ~r = 0 and the relative OLS
Given OLS estimator we can compute the vector of differences R
~~
estimate R
b ~r = 0.
~~
We consider the sampling distribution of : R
b ~r .
~~
The general test for all restriction is based on the sampling distribution of R
b under the null
hypothesis.
Prof. Rizzi Laura
Outline

~~
R
b has sampling distribution given by:
E (Rb)
var (Rb)
=
=
R
h
i
h
i
T
T T
E (Rb R)(Rb R)
= E R(b )(b ) R
h
i
T
T
T
2
T
1 T
RE (b )(b ) R = Rvar (b)R = R(X X ) R
~
b is a function of ~
u then the sampling distribution of Rb depends on the distribution of ~
u which is
assumed u N(0, 2 ). Thus:
b
Rb
Rb R
N(, (X X )
2
)
1
N(R, R(X X )
2
N(0, R(X X )
R )
R )
If R = r under the true H0 then we obtain that:

2
(Rb r ) N(0, R(X X )
Prof. Rizzi Laura
R )
Outline

Remember that:
2
x N(0, )
N(0, 1)
x2
2
Considering the previous random variable, squared and standardized, we have:

(Rb r )T (Rb r )
2
(q)
2 R(X T X )1 R T
(Rb r )
R(X X )
i1
(Rb r ) (q)
But often is unknown and has to be estimated:

eT e
2
(n k)
2
Prof. Rizzi Laura
Outline

The ratio between two independent r.v. x1 and x2 with probability distribution 2 with d.f. n1 and
n2 , respectively, divided by their d.f., drives a r.v. with probability distribution F . This r.v. F
under the null hypothesis is distributed as follows:
h
i1
(Rb r )/q
(Rb r )T R(X T X )1 R T
F =
F (q, n k)
e T e/(n k)
The null H0 : R = r is rejected if the absolute sampling value of the test F is greater than the
critical values (given a significance level equal to 5% or 1%) on the F (q, n k) distribution.
We furthermore know that:
eT e
2
s =
nk
Then we obtain:
F = (Rb r )
h
i
2
T
1 T 1
s R(X X ) R
(Rb r )/q F (q, n k)
Where s 2 (X T X )1 is the variance-covariance estimated matrix of b.
Prof. Rizzi Laura
Outline

Examples:
[1]- H0 : j = 0
Rb =
b
1
.
.
.
0
.
.
bj .
bk
= bj
With r = 0 and q = 1. Then Rb r = bj and:
R(X X )
c11
..
.
cjj
..
.
ckk
Prof. Rizzi Laura
0
.
.
.
1
.
.
.
0
= cjj
Outline

Where R is (1 k) then R T is (k 1) and (X T X )1 is (k k).
The test F is the following:
F =
Or: t =
F =
bj
s.e.(bj )
bj2
s 2 cjj
bj2
F (1, n k)
var (bj )
t(n k), this is the test t for the hypothesis on the significance of the
parameter j .
[2]- H0 : j = 1
This test is similar to the previous one, then:
t=
bj 1
t(n k)
s.e.(bj )
F =
(bj 1)2
var (bj )
Where bj 1 = Rb r .
Prof. Rizzi Laura
Outline

[3]- H0 : 2 3 = 1
b1
b2
b3
0
.
..
bk
Rb =
= b2 b3
Furthermore r = 1.
c11
c21
c31
0
.
..
R(X X )
c12
c22
c32
.
.
.
c33
.
.
.
0
1
1
.
..
0
ckk
= c22 2c23 + c33
Prof. Rizzi Laura
Outline

Then:
F =
(b2 b3 1)2
(b2 b3 1)2
=
F (1, n k)
s 2 (c22 2c23 + c33 )
var (b2 ) 2cov (b2 b3 ) + var (b3 )
(b2 b3 1)
t= p
t(n k)
var (b2 b3 )
p
The confidence interval for 2 3 is [(b2 b3 ) t0.025 var (b2 b3 )].
[4]- H0 : 3 = 4 o H0 : 3 + 4 = 0
From above we derive:
t= p
(b3 + b4 )
var (b3 + b4 )
t(n k)
Then Rb = b3 + b4 and r = 0.
H0 : 2 = 3 = = k = 0
Prof. Rizzi Laura
Outline

In this case q = k 1, there is not a single hypothesis but a multiple one which relates to k 1
regression coefficients (in this case the test t is not equivalent to the test F ). We consider in this
situation a partition of the X matrix:

~ = ~i
X
X~2
~
Where X2 is the (n (k 1)) matrix of the observations on the covariates, while ~i is the (n 1)
vector with all values equal to 1 relative to the constant.

We consider also the partition of the vector ~
b as follows ~
bT =
b1 b2T
1
0
Rb = Rb2 =
.
.
.
0
..
.
0
0
.
.
.
b2
b3
.
.
.
0
bk
1
= b~2
This a ((k 1) 1) vector of estimated parameters
Prof. Rizzi Laura
Outline

The partitioned matrix is X =
T
X X =

X2
iT
X2T

, then:
X2
n
X2T i
i T X2
X2T X2
The inverse of the partitioned matrix is:

T
(X X )

=

B22
h
i1
h

Where B22 = X2T X2 X2T in1 i T X2
= X2T X2 I
i1
h
= (X T X )1
B22 = X2T AX2
1 T
n ii
i1
and then
Where A is the idempotent matrix such that AAT = A, from which we obtain:
Prof. Rizzi Laura
Outline

T
R(X X )
= (X
X )
Which is a ((k 1) (k 1)) matrix equal to B22 . Then the test F is:
h
i1
(Rb r )/q
(Rb r )T R(X T X )1 R T
F
e T e/(n k)
F (q, n k)
~
b2 /(k 1)
b2T (X T X )~
F (k 1, n k)
e T e/(n k)
This is the test on the significance of all regression coefficients:

F =
ESS/(k 1)
F (k 1, n k)
RSS/(n k)
Prof. Rizzi Laura
Outline

ESS
=1
Given that R 2 = TSS
test F as follows:
RSS
TSS
F =
, ESS = R 2 TSS and RSS = (1 R 2 )TSS we may express the

R 2 /(k 1)
F (k 1, n k)
(1 R 2 )/(n k)
We consider the null hypothesis H0 : ~2 = 0

This hypothesis relates to a sub-set of regression coefficients and we consider a partition of the X
matrix and of the vector :
Y =
X1
X2
b1
b2
Prof. Rizzi Laura

+ e = X 1 b1 + X 2 b2 + e
Outline

Where X1 is a (n k1 ) matrix , X2 is a (n k2 ) or (n (k k1 )) matrix, b1 is a (k1 1) vector
and b2 is a (k2 1) vector.
In this case we have:

X1T X1 X1T X2
T
X X =
X2T X1 X2T X2
We may prove that: R(X T X )1 R T = (X2T M1 X2 )1 . Where M1 = I X1 (X1T X1 )1 X1T is a
projection matrix such that M1 X1 = 0, M1 e = e and M1 Y = e1 which are the residuals of the
regression of Y only on X1 .
Then:
h
i
T
T
1 T 1
T
T
(Rb r ) R(X X ) R
(Rb r )/q = b2 (X2 M1 X2 )b2 /k2
Where k2 is the number of restrictions.
Prof. Rizzi Laura
Outline

Moreover, given that:
M1 Y = M1 X1 b1 + M1 X2 b2 + M1 e = M1 X2 b2 + e
Considering the quadratic form we obtain:
T
(M1 Y ) (M1 Y ) = Y M1 Y = b2 X2 M1 X2 b2 + e e
Where:
I Y T M1 Y are the RSS of the regression of Y on X1 ;
I e T e are the RSS of the regression of Y on X1 and X2 ;
I b2T X2T M1 X2 b2 is the reduction of the RSS when the variable X2 is included into the
regression of Y on X1 ;

X1 X2
We may denote with e T e the RSS of the regression of Y on X =
and with e T e
the RSS dof the regression of Y only on X1 .
Prof. Rizzi Laura
Outline

Then:
e
T T
b2 X 2 M 1 X 2 b2
The test statistic is:

F =
=
=
Y M1 Y = b2 X2 M1 X2 b2 + e e
e
e e e
(e T e e T e)/k2
F (k2 , n k)
e T e/(n k)
Prof. Rizzi Laura

Test PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Test PDF

Uploaded by

Copyright:

Available Formats

Outline

Test on single parameters

Test on regression parameters

October 28, 2013

Prof. Rizzi Laura

Test on regression parameters

Test on single parameters

Test on more parameters

Test: matrix algebra approach

Prof. Rizzi Laura

Test on regression parameters

Test on single parameters

Prof. Rizzi Laura

Test on regression parameters

Test on more parameters

Prof. Rizzi Laura

Test on regression parameters

Test on more parameters

The decision rule could be:

Prof. Rizzi Laura

Test on regression parameters

Test on more parameters

PrH0 [|t2 | 1.96, |t3 | > 1.96]

PrH0 [|t2 | > 1.96] PrH0 [|t3 | > 1.96] +

PrH0 [|t2 | > 1.96] PrH0 [|t3 | 1.96] +

PrH0 [|t2 | 1.96] PrH0 [|t3 | > 1.96]

With independent t2 and t3 we obtain:

Prof. Rizzi Laura

Test on regression parameters

Test on more parameters

The test F derived in Stock and Watson (see 5.7)

Prof. Rizzi Laura

Test on regression parameters

F distribution in large samples

t12 + t22 2t1 ,t2 t1 t2

Prof. Rizzi Laura

Test on regression parameters

Stata software example

reg testscr str expn_stu pctel, r;

The test command follows the regression

There are q=2 restrictions being tested

The 5% critical value for q=2 is 3.00

Prof. Rizzi Laura

Test on regression parameters

Test: matrix algebra approach

Prof. Rizzi Laura

Test on regression parameters

Test: matrix algebra approach

Prof. Rizzi Laura

Test on regression parameters

Test: matrix algebra approach

Prof. Rizzi Laura

Test on regression parameters

Test: matrix algebra approach

Prof. Rizzi Laura

Test on regression parameters

Test: matrix algebra approach

Prof. Rizzi Laura

Test on regression parameters

Test: matrix algebra approach

Prof. Rizzi Laura

Test on regression parameters

Test: matrix algebra approach