Professional Documents
Culture Documents
Regression Models
Simple
1 order
Multiple
1 order
2 order
Higher order
Interaction
2 order
Higher order
E(Y)=0+ 1x+ 2 x2
Interpretation of model parameters:
0: y-intercept. The value of E(Y) when x1 = x2 = 0
1 : is the shift parameter;
2 : is the rate of curvature;
75.00
50.00
25.00
0.00
2.00
4.00
6.00
8.00
10.00
Model 1: E(Y) = 0 + 1x
Model Summary
Model
1
R
,973a
R Square
,947
a. Predictors: (Constant), x
Model
1
Regression
Residual
Total
Sum of
Squares
80624,915
4500,202
85125,117
a. Predictors: (Constant), x
Adjusted
R Square
,947
ANOVAb
df
1
103
104
Mean Square
80624,915
43,691
F
1845,332
Sig.
,000a
Coefficientsa
b. Dependent Variable: y
Unstandardized
Coefficients
Model
B
Std. Error
1
(Constant)
-19,959
1,483
x
10,744
,250
a. Dependent Variable: y
Std. Error of
the Estimate
6,60994
Standardized
Coefficients
Beta
,973
t
-13,454
42,957
Sig.
,000
,000
Linear Regression
100.00
y = -19.96 + 10.74 * x
R-Square = 0.95
75.00
50.00
25.00
0.00
2.00
4.00
6.00
8.00
10.00
R
,996a
R Square
,991
Adjusted
R Square
,991
Model
1
Regression
Residual
Total
Sum of
Squares
84381,422
743,695
85125,117
Std. Error of
the Estimate
2,68707
ANOVAb
df
1
103
104
Mean Square
84381,422
7,220
F
11686,632
Sig.
,000a
Coefficientsa
Model
1
(Constant)
XSquare
Unstandardized
Coefficients
B
Std. Error
2,340
,417
,997
,009
a. Dependent Variable: y
Standardized
Coefficients
Beta
,996
t
5,608
108,105
Sig.
,000
,000
Linear Regression
100.00
75.00
50.00
25.00
0.00
0.00
25.00
50.00
XSquare
75.00
100.00
R
.996a
R Square
.991
Adjusted
R Square
.991
Std. Error of
the Estimate
2.66608
Regression
Residual
Total
Sum of
Squares
84400.103
725.014
85125.117
df
2
102
104
Mean Square
42200.052
7.108
F
5936.999
Sig.
.000a
Model
1
(Constant)
x
XSquare
Coefficientsa
Unstandardized
Coefficients
B
Std. Error
4.177
1.206
-.830
.512
1.071
.046
a. Dependent Variable: y
Standardized
Coefficients
Beta
-.075
1.069
t
3.463
-1.621
23.046
Sig.
.001
.108
.000
Regression Models
Simple
1 order
Multiple
1 order
2 order
Higher order
Interaction
2 order
Higher order
>0
3
<0
3
Regression Models
Simple
1 order
Multiple
1 order
2 order
Higher order
Interaction
2 order
Higher order
A bivariate model
E(Y)=0+1x1+2 x2
Changing x2 changes only the y-intercept.
A bivariate model
Y
Response
P la n e
X1
Y i = 0 + 1X 1i + 2X 2i + i
(( OO bb ss ee rr vv e dd YY ))
00
X2
( X 1 i , X 22 i )
E ( Y ) = 0 + 11 X 1 i + 2 X 2 i
Data: ExecSal.sav
Do not consider
x3
Modello
R-quadrato
R
R-quadrato corretto
,870a
,757
,747
a. Predittori: (Costante), Corporate assets (in million $), Years of Experience, Years of Education,
Number of Employees supervised
Simple regression
Multiple regression
Riepilogo del modello
Modello
R
1
dimension0
R-quadrato
,783a
,613
.
Predittori: (Costante), Years of Experience
R-quadrato
corretto
,609
Deviazione
standard Errore
della stima
15760,006
Coefficient of determination
1
Total variation
SST
SST
( yi y ) 2
i 1
SST (Total)
( y i y ) 2
i 1
SSR (Regression)
( yi y i ) 2
i 1
SSE (Error)
A solution: Adjusted R2
Each additional variable reduces adjusted R2, unless
SSE varies enough to compensate
Ra2
n 1 SSE
SSE
2
1
1
SST
SST
2
i
SSE
s
n k 1 n k 1
2
Model
Coefficienti non
standardizzati
Variables
1
B
(Costante)
Years of
Experience
Years of
Education
Number of
Employees
supervised
Corporate
assets (in
million $)
Deviazione
standard
Errore
Coefficienti
standardizz
ati
Beta
T-tests
t
Sig.
-37082,148
17052,089
-2,175
,032
2696,360
173,647
,785 15,528
,000
2656,017
563,476
,243
4,714
,000
41,092
7,807
,272
5,264
,000
244,569
83,420
,149
2,932
,004
Anova table
Anovab
Modello
1
Somma dei
quadrati
F-statistic
Media dei
quadrati
df
Regressione
4,766E10
Residuo
1,529E10
95
Totale
6,295E10
99
4 1,192E10 74,045
Sig.
,000a
1,609E8
. Predittori: (Costante), Corporate assets (in million $), Years of Experience, Years of
Education, Number of Employees supervised
a
df = k: number of
b. Variabile dipendente: Annual salary in $
regression slopes
p-vale of F-test
df = n-1: n=
number of
observations
MSE (mean
square error),
the estimate of
variance
Decision: reject
H0, i.e. accept
this model
Data summaries
Descriptive Statistics
Minimu Maximu Mean
Std.
N
Skewness
Kurtosis
m
m
Deviatio Statistic Std. Error Statistic Std. Error
Statistic Statistic
Statistic
Statistic Statistic
n
Age
32
108
194 144.94 27.395
.216
.414
-1.323
.809
Bidders
32
5
15
9.53
2.840
.420
.414
-.788
.809
Price
32
729
2131 1326.88 393.487
.396
.414
-.727
.809
Valid N (listwise)
32
Bivariate scatter-plots
2000
2000
1600
1200
1200
800
1600
800
120
140
160
Age
180
10
Bidders
12
14
R
R Square
a
.945
.892
Adjusted
R Square
.885
Std. Error of
the Estimate
133.485
Model
1
Regression
Residual
Total
Sum of
Squares
4283062.960
516726.540
4799789.500
df
2
29
31
Mean Square
2141531.480
17818.157
F
120.188
Sig.
.000a
Model
1
(Constant)
Age
Bidders
Unstandardized
Coefficients
B
Std. Error
-1338.951
173.809
12.741
.905
85.953
8.729
Standardized
Coefficients
Beta
.887
.620
t
-7.704
14.082
9.847
Sig.
.000
.000
.000
R
R Square
a
.977
.954
Adjusted
R Square
.949
Std. Error of
the Estimate
88.915
Model
1
Regression
Residual
Total
Sum of
Squares
4578427.367
221362.133
4799789.500
df
3
28
31
Mean Square
1526142.456
7905.790
F
193.041
Sig.
.000a
t
1.086
.432
-3.120
6.112
Sig.
.287
.669
.004
.000
Model
1
(Constant)
Age
Bidders
AgeBid
Unstandardized
Coefficients
B
Std. Error
320.458
295.141
.878
2.032
-93.265
29.892
1.298
.212
Standardized
Coefficients
Beta
.061
-.673
1.369
b2 + b3x1
Note: b = ^
E(Y) = 0+ 1x
E(Y) = 0+ 1 if x =1, i.e. Male
E(Y) = 0 if x =0, i.e. Female
0 is the base level, i.e Female is the reference category
1 is the additional effect if Male
In this simple model, only the means for the two groups are
modeled
x1 = 1 level A, x1 = 0 if not
x2 = 1 level B, x2 = 0 if not
Interpreting s
0 = C
1 = A - C
2 = B - C
Female group
Regression Output
Model Summary
Model
1
R
R Square
a
.392
.153
Adjusted
R Square
.145
Model
1
Regression
Residual
Total
Std. Error of
the Estimate
23320.282
ANOVAb
Sum of Squares
9651865066.845
53295882433.156
62947747500.001
df
Mean Square
9651865066.845
543835535.032
1
98
99
F
17.748
Sig.
.000a
Coefficientsa
(Constant)
Gender
Unstandardized
Coefficients
B
Std. Error
83847.059
3999.395
20739.305
4922.915
Standardized
Coefficients
Beta
.392
t
20.965
4.213
Sig.
.000
.000
It seems that
the two groups
are separated
Model 2 considers
same slope but
different
intercepts
Model Summary
Model
1
R
R Square
a
.860
.740
Adjusted
R Square
.735
Std. Error of
the Estimate
12981.615
ANOVA
Model
1
Regression
Residual
Total
Sum of Squares
46601081714.527
16346665785.474
62947747500.001
df
2
97
99
Mean Square
23300540857.264
168522327.685
F
138.264
Sig.
.000a
(Constant)
Gender
Years of Experience
Unstandardized
Coefficients
B
Std. Error
50614.312
3161.279
18894.215
2743.253
2633.831
177.875
Coefficients
Standardized
Coefficients
Beta
.357
.767
t
16.011
6.888
14.807
Sig.
.000
.000
.000
Model 3 considers
different slope
and different
intercepts
R
R Square
.868a
.754
Adjusted
R Square
.746
Std. Error of
the Estimate
12700.080
(Constant)
Gender
Years of Experience
ExpGender
Unstandardized
Coefficients
B
Std. Error
58049.768 4461.179
7798.504 5497.470
2044.541
308.565
864.122
373.653
Standardized
Coefficients
Beta
.147
.595
.301
t
13.012
1.419
6.626
2.313
Sig.
.000
.159
.000
.023
Estimated lines:
Y^ = 58049.8 + 2044.5*(Years of Experience) for female
^ = 65848.3 + 2908.7*(Years of Experience) for male
Y
x1 = Years of experience
x3 = Gender (1 if Male)
Note: x32 = x3 since
it is a dummy
Model 4
Model 5
Model 5
then
If x3 = 0 (female) then
E(Y) = 0 + 1x1 + 4x12
If x3 = 1 (male)
then
,875a
R-quadrato
corretto
R-quadrato
,766
Deviazione
standard Errore
della stima
,754
12507,735
Anovab
Modello
1
Somma
dei
quadrati
Media dei
quadrati
df
Regressione
4,824E10
Residuo
1,471E10
94
Totale
6,295E10
99
9,648E9 61,673
1,564E8
Sig.
,000a
Coefficienti non
standardizzati
Deviazion
e
standard
Errore
Beta
Sig.
(Costante)
52391,973 6497,971
8,063
,000
Years of
Experience
Gender
ExpGen
ExpSqu
Exp2Gen
3373,970 1165,248
,982 2,895
,005
21122,152 8285,802
-2081,897 1459,842
-53,181
45,001
112,836
54,950
,399
-,724
-,422
,904
2,549
-1,426
-1,182
2,053
,012
,157
,240
,043
To test
H0: g+1 = = k = 0
H1: at least one of the parameters being tested is not 0
Compute
( SSE R SSEC ) /( k g )
F
MSEC
Model 3:
E(Y) = 0 + 1x1 + 2x3 + 3x1x3
Model 5:
E(Y) = 0 + 1x1 + 2x3 + 3x1x3 + 4x12 + 5x3x12
Apply the F-test for H0: 4 = 5 = 0
Computer output
Variabili inserite/rimossec
Modello
Variabili
Variabili
inserite
rimosse Metodo
1
Exp2Gen,
.
Per
Gender, Years
blocchi
of Experience,
ExpSqu,
ExpGena
2
.a
Exp2Gen, Rimuovi
ExpSqub
F-statistic
F p-value
Variazione dell'adattamento
Model
RDeviazione
R- quadrat standard Variazione
quadr
o
Errore della
di RVariazio
ato corretto
stima
quadrato ne di F df1
,875
,766
,754 12507,735
,868b
,754
,746 12700,080
,766 61,673
-,012
2,488
df2
Sig.
Variazio
ne di F
94
,000
94
,089
Scatter plots
16.0
16.0
12.0
Cost of shipment
12.0
8.0
8.0
4.0
0.00
4.0
2.00
4.00
6.00
8.00
50
100
150
200
250
Distance shipped
Scatter plots in multiple regression often do not show too much information
R
.997a
R Square
.994
Adjusted
R Square
.992
Std. Error of
the Estimate
.4428
Model
1
Regression
Residual
Total
Sum of
Squares
449.341
2.745
452.086
df
5
14
19
Mean Square
89.868
.196
F
458.388
Sig.
.000a
Sig.
.259
.004
.623
.001
.513
.000
R
.997a
R Square
.994
Adjusted
R Square
.992
Std. Error of
the Estimate
.4346
Model
1
Regression
Residual
Total
Sum of
Squares
449.252
2.833
452.086
df
4
15
19
Mean Square
112.313
.189
F
594.623
Sig.
.000a
(Constant)
Weight of parcel in lbs.
Distance shipped
Weight squared
Weight*Distance
Coefficients
B
Std.
.475
-.578
.009
.087
.007
Error
.458
.171
.003
.019
.001
Standardized
Coefficients
Beta
-.300
.141
.369
.842
t
1.035
-3.387
3.421
4.485
11.753
Sig.
.317
.004
.004
.000
.000
Data: Express.sav
ANOVA Tables
Full model
Model
1
Regression
Residual
Total
ANOVAb
Sum of
Squares
449.341
2.745
452.086
df
5
14
19
Mean Square
89.868
.196
F
458.388
Sig.
.000a
Reduced model
ANOVAb
Model
1
Regression
Residual
Total
Sum of
Squares
445.452
6.633
452.086
df
3
16
19
Mean Square
148.484
.415
F
358.154
Sig.
.000a
F-statistic
To test H0: 4 = 5 = 0, from the ANOVA tables we have
F
9.92
MSEC
0.196
Computer output
Variables Entered/Removedc
Model
1
Variables
Entered
Weight*
Distance,
Distance
squared,
Weight
squared,
Weight of
parcel in
lbs.,
Distancea
shipped
Variables
Removed
Method
2
.
Distance
squared,
Weight b
squared
Enter
F-statistic
Remove
F p-value
Model Summary
Change Statistics
Model
1
2
R
.997a
.993b
R Square
.994
.985
Adjusted
R Square
.992
.983
Std. Error of
the Estimate
.4428
.6439
R Square
Change
.994
-.009
F Change
458.388
9.917
df1
df2
5
2
14
14
Sig. F Change
.000
.002
a. Predictors: (Constant), Weight*Distance, Distance squared, Weight squared, Weight of parcel in lbs., Distance shipped
b. Predictors: (Constant), Weight*Distance, Weight of parcel in lbs., Distance shipped
Reject H0: 4 = 5 = 0
Modello
R
R-quadrato
,963a
R-quadrato
corretto
,927
,922
Errore della
stima
7020,089
a. Predittori: (Costante), Corporate assets (in million $), Years of Experience, Years of Education, Gender, Number of
Employees supervised, ExpGender
Anovab
Somma
dei
quadrati
Model
1
Regressione
Residuo
Totale
Media dei
quadrati
df
5,836E10
4,583E9
93
6,295E10
99
Sig.
a. Predittori: (Costante), Corporate assets (in million $), Years of Experience, Years of Education, Gender, Number of Employees supervised, ExpGender
Model
Coefficienti non
standardizzati
B
(Costante)
Years of Experience
Gender
ExpGender
Years of Education
Number of Employees
supervised
Corporate assets (in million
$)
Deviazion
e standard
Errore
-38331,331 9533,238
2178,964
171,979
13203,101 3137,775
669,546
209,042
2689,594
311,914
53,239
4,470
180,310
46,600
Coefficient
i
standardiz
zati
Beta
,634
,249
,233
,246
,353
,110
Sig.
-4,021
,000
12,670
,000
4,208
,000
3,203
,002
8,623
,000
11,910
,000
3,869
,000
Predictors
Adj. R2
Standard
error
0.747 12685.31
x1, x3
0.735 12981.62
138.26
0.746 12700.08
98.09
0.922
7020.09
F-stat
74.05
197.38