Introduction to Econometrics

Lecture 4
Multiple regression models

INEMET [U13783]

Guy Judge March 2007

Lecture plan
the need for additional regressors
classical assumptions and least squares
estimation extended to allow for several
regressors
interpreting the results
t tests of individual parameter values
R squared and R bar squared
Analysis of variance and F tests
practical illustrations
INEMET [U13783]

Guy Judge March 2007

multiple regression models: examples

sales-advertising equations extended to include
variables such as consumers income, price and
the price and advertising of competitors
products
earnings equations extended to include
experience or age and other variables in addition
to years of schooling (including dummy variables
to examine the importance of gender, ethnic
group etc)
INEMET [U13783]

Guy Judge March 2007

Assumptions of the multiple linear regression model (1)

A stable linear stochastic relationship

Yi = 0 + 1 X1i + 2 X2i +....+ k Xki + ui

for i = 1,2,...,n

The parameters are the same for each observation no

structural differences

INEMET [U13783]

Guy Judge March 2007

Assumptions of the multiple linear regression model (2)

u is a random variable distributed with

(a) zero mean
E(ui) = 0 for all i
(b) constant variance
var(ui) = u2 for all i
(b) implies disturbances are homoskedastic

INEMET [U13783]

Guy Judge March 2007

Assumptions of the multiple linear regression model (3)

Disturbances are independent of one

another
E(ui|uj ) = 0 for i j
Note: independence means E(ui|uj ) = E(ui )
which = 0 by assumption 2a
INEMET [U13783]

Guy Judge March 2007

Assumptions of the multiple linear regression model (4)

Disturbances are independent of each of

the X variables
E(ui|Xj) = E(ui) = 0 for all i, j

INEMET [U13783]

Guy Judge March 2007

Assumptions of the multiple linear regression model (5)

u has a normal distribution

thus with (2) we can write
u ~ N(0,u2)

INEMET [U13783]

Guy Judge March 2007

Assumptions of the multiple linear regression model (6)

There is no exact linear relationship among

the X variables
(they are linearly independent)

INEMET [U13783]

Guy Judge March 2007

Least squares estimators

INEMET [U13783]

Guy Judge March 2007

Least squares estimators matrix notation

INEMET [U13783]

Guy Judge March 2007

Tests of significance of the individual

parameter estimates (1)

INEMET [U13783]

Guy Judge March 2007

Tests of significance of the individual

parameter estimates (2)

INEMET [U13783]

Guy Judge March 2007

Goodness of fit
The coefficient of multiple determination is defined as
2
2

e
R2 2 = 1 - 2
y
y

Note: this is no longer the square of a simple correlation

coefficient.

INEMET [U13783]

Guy Judge March 2007

Adjusted R squared (R bar squared)

INEMET [U13783]

Guy Judge March 2007

More on R bar squared

INEMET [U13783]

Guy Judge March 2007

Testing the overall significance of the regression.

Analysis of variance (F test)

INEMET [U13783]

Guy Judge March 2007

R squared and the F value

It is possible to calculate an F value from the value of

R squared

INEMET [U13783]

Guy Judge March 2007

more on the extended sales model

sales = 0+ 1price+ 2income+ 3pcompet+ 4ads + u
Intercept
PRICE
INCOME
PCOMPE T
ADS

Coefficients Standard Error

t Stat
P-value
-30.5565 9.549212555
-3.1999 0.015068
-2.84248 0.597282084 -4.75903 0.002062
0.088978 0.034783217 2.558074 0.037657
1.995104 0.672915515 2.964866 0.02096
1.16726 0.162942327 7.16364 0.000183

RegressionStatistics
0.99684
MultipleR
0.99369
RSquared
AdjustedRSquared
0.99009
StandardError
1.26813
12
Observations
ANOVA
Regression
Residual
Total

INEMET [U13783]

df
SS
MS
F
Significance F
4 1773.66 443.4149 275.7299 8.92072E-08
7 11.25705 1.60815
11 1784.917

Guy Judge March 2007

more on the extended earnings model

earnings = 1+ 2s+ 3exper + u

INEMET [U13783]

Guy Judge March 2007