Professional Documents
Culture Documents
Chapter 11
Simple Regression
Chap 12-1
Chapter Goals
After completing this chapter, you should be
able to:
Explain the correlation coefficient and perform a
hypothesis test for zero population correlation
Explain the simple linear regression model
Obtain and interpret the simple linear regression
equation for a set of data
Describe R2 as a measure of explanatory power of the
regression model
Understand the assumptions behind regression
analysis
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-2
Chapter Goals
(continued)
Chap 12-3
Correlation Analysis
Correlation analysis is used to measure
strength of the association (linear relationship)
between two variables
Correlation is only concerned with strength of the
relationship
No causal effect is implied with correlation
Chap 12-4
Correlation Analysis
The population correlation coefficient is
denoted (the Greek letter rho)
The sample correlation coefficient is
r
where
s xy
s xy
sxsy
(x x)(y y)
n 1
Chap 12-5
H0 : 0
r (n 2)
(1 r )
Chap 12-6
Decision Rules
Hypothesis Test for Correlation
Lower-tail test:
Upper-tail test:
Two-tail test:
H0: 0
H1: < 0
H0: 0
H1: > 0
H0: = 0
H1: 0
-t
r (n 2)
(1 r )
2
/2
-t/2
/2
t/2
has n - 2 d.f.
Chap 12-7
Introduction to
Regression Analysis
Regression analysis is used to:
Predict the value of a dependent variable based on
the value of at least one independent variable
Explain the impact of changes in an independent
variable on the dependent variable
Dependent variable: the variable we wish to explain
(also called the endogenous variable)
Chap 12-8
Yi 0 1x i i
Where 0 and 1 are the population model
coefficients and is a random error term.
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-9
Population
Slope
Coefficient
Independent
Variable
Random
Error
term
Yi 0 1Xi i
Linear component
Random Error
component
Chap 12-10
Yi 0 1Xi i
Observed Value
of Y for Xi
Predicted Value
of Y for Xi
Slope = 1
Random Error
for this Xi value
Intercept = 0
Xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
X
Chap 12-11
Estimate of
the regression
Estimate of the
regression slope
intercept
y i b0 b1x i
Value of x for
observation i
ei ( y i - y i ) y i - (b0 b1x i )
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-12
Chap 12-13
b1
(x x)(y y)
i 1
x
2
(x
x
)
i
rxy
sY
sX
i 1
b0 y b1x
The regression line always goes through the mean x, y
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-14
Chap 12-15
E[ i ] 0 and E[ i ] 2
for (i 1, , n)
Chap 12-16
Interpretation of the
Slope and the Intercept
b0 is the estimated average value of y
when the value of x is zero (if x = 0 is
in the range of observed x values)
Chap 12-17
Chap 12-18
Square Feet
(X)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Chap 12-19
Graphical Presentation
House price model: scatter plot
Chap 12-20
Chap 12-21
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-22
Chap 12-23
Graphical Presentation
House price model: scatter plot and
regression line
Slope
= 0.10977
Intercept
= 98.248
Chap 12-24
Interpretation of the
Intercept, b0
house price 98.24833 0.10977 (square feet)
b0 is the estimated average value of Y when the
value of X is zero (if X = 0 is in the range of
observed X values)
Here, no houses had 0 square feet, so b0 = 98.24833
just indicates that, for houses within the range of
sizes observed, $98,248.33 is the portion of the
house price not explained by square feet
Chap 12-25
Interpretation of the
Slope Coefficient, b1
house price 98.24833 0.10977 (square feet)
Chap 12-26
Measures of Variation
Total variation is made up of two parts:
SST
SSR
SSE
Total Sum of
Squares
Regression Sum
of Squares
Error Sum of
Squares
SST (y i y)2
SSR (y i y)2
SSE (y i y i )2
where:
Chap 12-27
Measures of Variation
(continued)
Chap 12-28
Measures of Variation
(continued)
Y
yi
2
SSE = (yi - yi )
_
y
xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
_
y
X
Chap 12-29
Coefficient of Determination, R2
The coefficient of determination is the portion
of the total variation in the dependent variable
that is explained by variation in the
independent variable
The coefficient of determination is also called
R-squared and is denoted as R2
SSR regression sum of squares
R
SST
total sum of squares
2
note:
0 R 1
Chap 12-30
Examples of Approximate
r2 Values
Y
r2 = 1
r2 = 1
r =1
2
Chap 12-31
Examples of Approximate
r2 Values
Y
0 < r2 < 1
X
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-32
Examples of Approximate
r2 Values
r2 = 0
No linear relationship
between X and Y:
r2 = 0
Chap 12-33
Excel Output
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
SSR 18934.9348
R
0.58082
SST 32600.5000
2
Regression Statistics
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-34
Chap 12-35
Correlation and R2
The coefficient of determination, R2, for a
simple regression is equal to the simple
correlation squared
R r
2
2
xy
Chap 12-36
Estimation of Model
Error Variance
An estimator for the variance of the population model
error is
n
2
e
i
SSE
s
n2 n2
2
2
e
i1
Chap 12-37
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
s e 41.33032
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-38
Chap 12-39
small se
large se
Chap 12-40
2
2
(xi x) (n 1)s x
where:
sb1
SSE
se
n2
Chap 12-41
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
Observations
ANOVA
sb1 0.03297
41.33032
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-42
small Sb1
large Sb1
Chap 12-43
Test statistic
b1 1
t
sb1
d.f. n 2
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
where:
b1 = regression slope
coefficient
1 = hypothesized slope
sb1 = standard
error of the slope
Chap 12-44
Square Feet
(x)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Chap 12-45
H1: 1 0
Coefficients
Intercept
Square Feet
b1
Standard Error
sb1
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
b1 1 0.10977 0
t
3.32938
t
sb1
0.03297
Chap 12-46
H1: 1 0
Coefficients
Intercept
Square Feet
d.f. = 10-2 = 8
t8,.025 = 2.3060
/2=.025
Reject H0
/2=.025
Do not reject H0
-tn-2,/2
-2.3060
Reject H0
tn-2,/2
2.3060 3.329
b1
Standard Error
sb1
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
Decision:
Reject H0
Conclusion:
There is sufficient evidence
that square footage affects
house price
Chap 12-47
Standard Error
t Stat
P-value
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-48
Standard Error
t Stat
P-value
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-49
MSR
F
MSE
MSR
SSR
k
MSE
SSE
n k 1
Chap 12-50
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
MSR 18934.9348
F
11.0848
MSE 1708.1957
10
df
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
P-value for
the F-Test
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-51
Test Statistic:
H 0 : 1 = 0
MSR
F
11.08
MSE
H 1 : 1 0
= .05
df1= 1
df2 = 8
Decision:
Reject H0 at = 0.05
Critical
Value:
F = 5.32
Conclusion:
= .05
Do not
reject H0
Reject H0
F.05 = 5.32
Chap 12-52
Prediction
The regression equation can be used to
predict a value for y, given a particular x
For a specified value, xn+1 , the predicted
value is
y n1 b0 b1x n1
Chap 12-53
Predictions Using
Regression Analysis
Predict the price for a house
with 2000 square feet:
Chap 12-54
Risky to try to
extrapolate far
beyond the range
of observed Xs
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-55
y = b0+b1xi
Prediction Interval
for an single
observed y, given xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
xi
Chap 12-56
1 (x n1 x)2
2
n (x i x)
Chap 12-57
1 (x n1 x)2
1
2
n (x i x)
Chap 12-58
1
(x i x)2
317.85 37.12
2
n (x i x)
Chap 12-59
y n1 t n-1,/2s e
1
(Xi X)2
1
317.85 102.28
2
n (Xi X)
Chap 12-60
Chapter Summary
Introduced the linear regression model
Reviewed correlation and the assumptions of
linear regression
Discussed estimating the simple linear
regression coefficients
Described measures of variation
Described inference about the slope
Addressed estimation of mean values and
prediction of individual values
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-61
Chap 12-62