Professional Documents
Culture Documents
Chapter 11
Simple Regression
Ch. 11-1
Chapter Goals
After completing this chapter, you should be
able to:
Ch. 11-2
Chapter Goals
(continued)
Ch. 11-3
11.1
Ch. 11-4
y b0 b1x
Cov(x, y)
b1
s2x
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
b 0 y b1x
Ch. 11-5
Introduction to
Regression Analysis
Ch. 11-6
11.2
Yi 0 1x i i
Ch. 11-7
Population
Y intercept
Dependent
Variable
Independent
Variable
Random
Error
term
Yi 0 1Xi i
Linear component
Random Error
component
Ch. 11-8
Yi 0 1Xi i
Observed Value
of Y for Xi
Predicted Value
of Y for Xi
Slope = 1
Random Error
for this Xi value
Intercept = 0
Xi
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 11-9
Estimate of
the regression
Estimate of the
regression slope
intercept
y i b0 b1x i
Value of x for
observation i
ei ( y i - y i ) y i - (b0 b1x i )
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 11-10
11.3
Ch. 11-11
b1
(x x)(y y)
i1
2
(x
x
)
i
sy
Cov(x, y)
rxy
2
sx
sx
i1
b0 y b1x
Ch. 11-12
Ch. 11-13
E[ i ] 0 and E[ i ] 2
for (i 1, , n)
for all i j
Ch. 11-14
Interpretation of the
Slope and the Intercept
Ch. 11-15
Ch. 11-16
Square Feet
(X)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Ch. 11-17
Graphical Presentation
Ch. 11-18
Ch. 11-19
(continued)
Ch. 11-20
Excel Output
Ch. 11-21
Excel Output
(continued)
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Significance F
0.01039
Ch. 11-22
Graphical Presentation
Intercept
= 98.248
Ch. 11-23
Interpretation of the
Intercept, b0
house price 98.24833 0.10977 (square feet)
Ch. 11-24
Interpretation of the
Slope Coefficient, b1
house price 98.24833 0.10977 (square feet)
Ch. 11-25
11.4
Measures of Variation
SST
SSR
Total Sum of
Squares
Regression Sum
of Squares
SST (y i y)2
SSR (y i y)2
SSE
Error Sum of
Squares
SSE (y i y i )2
where:
= Average
value of the dependent variable
y
i
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 11-26
Measures of Variation
(continued)
Ch. 11-27
Measures of Variation
(continued)
Y
yi
2
SSE = (yi - yi )
_
y
xi
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 11-28
Coefficient of Determination, R2
SST
total sum of squares
2
note:
0 R 1
Ch. 11-29
Examples of Approximate
r2 Values
Y
r2 = 1
r2 = 1
r =1
2
Ch. 11-30
Examples of Approximate
r2 Values
Y
0 < r2 < 1
X
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 11-31
Examples of Approximate
r2 Values
r2 = 0
No linear relationship
between X and Y:
r2 = 0
Ch. 11-32
Excel Output
SSR 18934.9348
R
0.58082
SST 32600.5000
2
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Significance F
0.01039
Ch. 11-33
Correlation and R2
R r
2
2
xy
Ch. 11-34
Estimation of Model
Error Variance
2
e
i
SSE
s
n2 n2
2
2
e
i1
Ch. 11-35
Excel Output
s e 41.33032
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Significance F
0.01039
Ch. 11-36
small se
large se
Ch. 11-37
11.5
2
2
(xi x) (n 1)s x
where:
Ch. 11-38
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
sb1 0.03297
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Significance F
0.01039
Ch. 11-39
small Sb1
large Sb1
Ch. 11-40
Test statistic
b1 1
t
sb1
d.f. n 2
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
where:
b1 = regression slope
coefficient
1 = hypothesized slope
sb1 = standard
error of the slope
Ch. 11-41
Square Feet
(x)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Ch. 11-42
H1: 1 0
Coefficients
Intercept
Square Feet
sb1
b1
Standard Error
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
b1 1 0.10977 0
t
3.32938
t
sb1
0.03297
Ch. 11-43
H1: 1 0
Coefficients
Intercept
Square Feet
d.f. = 10-2 = 8
t8,.025 = 2.3060
/2=.025
Reject H0
/2=.025
Do not reject H0
-tn-2,/2
-2.3060
Reject H0
tn-2,/2
2.3060 3.329
sb1
b1
Standard Error
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
Decision:
Reject H0
Conclusion:
There is sufficient evidence
that square footage affects
house price
Ch. 11-44
P-value = 0.01039
H0: 1 = 0
H1: 1 0
Coefficients
Intercept
Square Feet
P-value
Standard Error
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
Standard Error
t Stat
P-value
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Ch. 11-46
Standard Error
t Stat
P-value
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Ch. 11-47
F Test statistic:
where
MSR
F
MSE
MSR
SSR
k
MSE
SSE
n k 1
Ch. 11-48
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
MSR 18934.9348
F
11.0848
MSE 1708.1957
With 1 and 8 degrees
of freedom
P-value for
the F-Test
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Significance F
0.01039
Ch. 11-49
Test Statistic:
H0: 1 = 0
MSR
F
11.08
MSE
H1: 1 0
= .05
df1= 1
df2 = 8
Decision:
Reject H0 at = 0.05
Critical
Value:
F = 5.32
Conclusion:
= .05
Do not
reject H0
Reject H0
F.05 = 5.32
Ch. 11-50
11.6
Prediction
y n1 b0 b1x n1
Ch. 11-51
Predictions Using
Regression Analysis
Predict the price for a house
with 2000 square feet:
Ch. 11-52
Risky to try to
extrapolate far
beyond the range
of observed Xs
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
Ch. 11-53
y = b0+b1xi
Prediction Interval
for an single
observed y, given xi
Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall
xi
Ch. 11-54
1 (x n1 x)2
2
n (x i x)
Ch. 11-55
1 (x n1 x)2
1
2
n (x i x)
Ch. 11-56
1
(x i x)2
317.85 37.12
2
n (x i x)
Ch. 11-57
y n1 t n-1,/2se
1
(Xi X)2
1
317.85 102.28
2
n (Xi X)
Ch. 11-58
11.7
Correlation Analysis
Correlation analysis is used to measure
strength of the association (linear relationship)
between two variables
Ch. 11-59
Correlation Analysis
r
where
s xy
s xy
sxsy
(x x)(y y)
n 1
Ch. 11-60
H0 : 0
r (n 2)
(1 r )
2
Ch. 11-61
Decision Rules
Hypothesis Test for Correlation
Lower-tail test:
Upper-tail test:
Two-tail test:
H0: 0
H1: < 0
H0: 0
H1: > 0
H0: = 0
H1: 0
-t
r (n 2)
(1 r )
/2
/2
-t/2
t/2
has n - 2 d.f.
Ch. 11-62
11.9
Graphical Analysis
Ch. 11-63
Chapter Summary
Ch. 11-64