Professional Documents
Culture Documents
Analysis U4320
10 20 30 40 50 60 70 X
Income
How well does the predicted line explain the variation in
the independent variable money spent?
I. Explaining Variation: R2
Total Variation
$ Spent on
Health Care
Y
Y-Y=deviation unexplained by regression
8 x x
(x,y)
7 x
x x Y-Y=deviation explained by regression
6
5.9 Y
x x Y-Y=total deviation around Y
5
4 x
Y=a+bx
10 20 30 40 50 60 70 X
Income
I. Explaining Variation: R2
Total Deviation
Y Y (Y Y ) (Y Y ) .
Total = Explained + Unexplained
Deviation Deviation Deviation
10 20 30 40 50 60 70 X
Income
Here, SSR = SST. The variance in the Y's is
completely explained by the regression line.
I. Explaining Variation: R2
On the other hand, if there is no relation
between X and Y:
Y
Money Spent 8 x
on Health Care 7 x
x x
6 Y
x x
5 x
4
10 20 30 40 50 60 70 X
Income
Now SSR is 0 and SSE=SST. The regression
line explains none of the variance in Y.
I. Explaining Variation: R2
3. Formula
So we can construct a useful statistic.
Take the ratio of the Regression Sum of Squares
perfect correlation.
For data with little relation, it's near 0.
SSR
R2
SST
5.90
189.54
.031
I. Explaining Variation: R2
c) Each of the 4 different models has an
associated R2.
Eq # 2: R2 = 0.035
Eq # 3: R2 = 0.039
Eq # 4 = R2 = .04
I. Explaining Variation: R2
F. Using R2 in Practice
1. Useful Tool
2. Measure of Unexplained Variance
3. Not a Statistical Test
4. Don't Obsess about R2
5. You can always improve R2 by adding
variables
I. Explaining Variation: R2
G. Example
You'll notice that R2 increase every time.
No matter what variables you add you can
always increase your R2.
II. Adjusted R2
II. Adjusted R2
A. Definition of Adjusted R2
So we'd like a measure like R2, but one that takes into
account the fact that adding extra variables always
increases your explanatory power.
The statistic we use for this is call the Adjusted R2, and
its formula is:
n 1
R2 1 (1 R2 );
nk
n number of observations,
k number of independent variables.
We calculate:
n 1
R2 1 (1 R2 )
nk
470 1
1 (1.03555)
470 3
.0314
II. Adjusted R2
C. Stepwise Regression
One strategy for model building is to add
variables only if they increase your adjusted R2.
This technique is called stepwise regression.
However, I don't want to emphasize this
approach to strongly. Just as people can fixate
on R2 they can fixate on adjusted R2.
****IMPORTANT****
If you have a theory that suggests that certain
variables are important for your analysis then
include them whether or not they increase the
adjusted R2.
Negative findings can be important!
III. F Tests
III. F Tests
A. When to use an F-Test?
Say you add a number of variables into a
regression model and you want to see if, as a
group, they are significant in explaining
variation in your dependent variable Y.
The F-test tells you whether a group of
variables, or even an entire model, is jointly
significant.
This is in contrast to a t-test, which tells whether an
individual coefficient is significantly different from
zero.
III. F Tests
B. Equations
To be precise, say our original equation is:
EQ 1: Y = b0 + b1X1 + b2X2,
and we add two more variables, so the new equation is:
EQ 2: Y = b0 + b1X1 + b2X2 + b3X3 + b4X4.
We want to test the hypothesis that
H0: b3 = b4 = 0.
That is, we want to test the joint hypothesis that X3 and X4
together are not significant factors in determining Y.
III. F Tests
C. Using Adjusted R2 First
There's an easy way to tell if these two
variables are not significant.
First, run the regression without X3 and X4 in it, then
run the regression with X3 and X4.
Now look at the adjusted R2's for the two
regressions. If the adjusted R2 went down, then X3
and X4 are not jointly significant.
So the adjusted R2 can serve as a quick test for
insignificance.
III. F Tests
D. Calculating an F-Test
If the adjusted R2 goes up, then you need to do a
more complicated test, F-Test.
1. Ratio
Let regression 1 be the model without X3 and X4, and
let regression 2 include X3 and X4.
The basic idea of the F statistic, then, is to compute
the ratio:
SSE1 SSE2
SSE2
III. F Tests
2. Correction
We have to correct for the number of independent
we add.
So the complete statistic is:
SSE1 SSE2
F m ;
SSE2
nk
m number of restrictions;
k number of independent variables.
b4MYSIGN + b5YELOWSTN
III. F Tests
1. Adding Extra Variables (cont.)
a) State the null hypothesis
H0: b4 = b5 = 0.
SSE1 SSE2
F m ,
SSE2
nk
III. F Tests
What is SSE1 --the sum of squared errors in the
first regression?
What is SSE2, the sum of squared errors in the
second regression?
m=2 N = 470 k=6
The formula is:
182.07 18182
.
F 2
18182
.
470 6
= 0.319
III. F Tests
c) Reject or fail to reject the null hypothesis?
The critical value at the 5 % level
2
F
470 6 from the table, is 3.00.
2
Is the F-statistic > F ?
470 6
If yes, then we reject the null hypothesis that the
variables are not significantly different from zero;
otherwise we fail to reject.
.319 < 3.00, so we can reject the null hypothesis.
that b1 = b2 = 0.