ECONOMETRICS I

(EKN309)

CHAPTER 2
TWO VARIABLE REGRESSION ANALYSIS:
SOME BASIC IDEAS
Chapter 1: the concept of regression in broad terms.
Chapter 2 and 3: introduction to the theory underlying the simplest possible
regression analysis the bivariate, or two-variable, regression:
where the dependent variable (the regressand) is related to a single
explanatory variable (the regressor).

Table 2.1 Weekly Family Income X, and Weekly Family Consumption

Expenditure Y, $

Figure 2.1 Conditional distribution of expenditure for various levels of

income (data of Table 2.1)

Definition of population regression curve:

Geometrically;
It is the locus of the conditional means of the dependent variable for the fixed
values of the explanatory variable(s).
More simply;
It is the curve connecting the means of the subpopulations of Y corresponding
to the given values of the regressor X.

2.2 THE CONCEPT OF POPULATION REGRESSION FUNCTION (PRF)

Each conditional mean is a function of

where is a given value of X.
=

. .

: some function of the explanatory variable, X. here, linear function.

In our example: is a linear function of .
(2.2.1): conditional expectation function (CEF) or population regression
function (PRF) or population regression (PR).
(2.2.1): the expected value of the distribution of Y given is functionally
related to .

(2.2.1): how the mean or average response of Y varies with X.

2.2 THE CONCEPT OF POPULATION REGRESSION FUNCTION (PRF)

(contd)

what form does the function assume?

It gives the functional form of the PRF however in real situations we do not
have the entire population available for examination.
So, its functional form is an empirical question.
i.e. The relationship btw consumption expenditure and income;
Assume that consumption expenditure is linearly related to income, so
the PRF, is assumed to be a linear function of :
= 1 + 2

(. . )

2.3 THE MEANING OF THE TERM LINEAR

Linearity in the variables:

-

The conditional expectation of Y is a linear function of .

The regression curve is a straight line.

Functional form examples?

Linearity in the parameters:

-

The conditional expectation of Y, , is a linear function of the

parameters, the s.

Functional form examples?

From now on the term linear regression will always mean a regression that is
linear in the parameters; the s (that is, the parameters are raised to the first
power only). It may or may not be linear in the explanatory variables, the Xs.

Table 2.3 Linear Regression Models

Model linear in parameters?

Model linear in variables?

YES

NO

YES

Linear regression model (LRM)

Linear regression model

(LRM)

NO

Nonlinear regression model

(NLRM)

Nonlinear regression model

(NLRM)

2.4 STOCHASTIC SPECIFICATION OF PRF

Figure 2.1: as family income increases, family consumption expenditure on

the average increases, too.
What about the consumption expenditure of an individual family in relation
to its (fixed) level of income?
Table 2.1 + Figure 2.1: the values.
Figure 2.1: given the income level of , an individual familys consumption
expenditure is clustered around the average consumption of all families at
that , that is, around its conditional expectation.

2.4 STOCHASTIC SPECIFICATION OF PRF (contd)

we can express the deviation of an individual around its expected value as

follows:
=
= +

(. . )

The deviation is an unobservable random variable positive or negative

values-.

: the stochastic disturbance OR stochastic error term.

HOW DO WE INTERPRET (2.4.1)?
The sum of two components: systematic (or deterministic) component AND
random (or nonsystematic) component.

2.4 STOCHASTIC SPECIFICATION OF PRF (contd)

If is linear in : = + (. . )
Substitute (2.2.2) into (2.4.1):

= +
= 1 + 2 +

. . that we are familiar with.

NOW, consider the individual consumption expenditures when = $80: the

following 5 equations constitute(2.4.3)

2.4 STOCHASTIC SPECIFICATION OF PRF (contd)

Take the expected value of (2.4.1) on both sides:

=
Since
itself:

= - because the expected value of a constant is that constant

= +

(. . )

AND since = ; then

= 0 (. . )
Thus, the assumption that the regression line passes through the conditional
means of Y implies that the conditional mean values of (conditional upon
the given Xs) are zero.

2.5 THE SIGNIFICANCE OF THE STOCHASTIC DISTURBANCE TERM

1. Vagueness of theory
ignorant or unsure about the other variables affecting Y; : as a substitute for
all the excluded or omitted variables from the model.
2. Unavailability of data
Even if we know what some of the excluded variables are, we may not have
quantitative information about them; captured in .
3. Core variables versus peripheral variables
The joint influence of all or some of the variables are so small and it is
meaningless to introduce them into the model explicitly; combined effect
being treated as a random variable, .

2.5 THE SIGNIFICANCE OF THE STOCHASTIC DISTURBANCE TERM (contd)

4. Intrinsic randomness in human behavior

Even if we succeed in introducing all the relevant variables into the model,
some intrinsic randomness in individual Ys that cannot be explained; s
reflecting this randomness.
5. Poor proxy variables
Errors in measurement where data may not be measured accurately;
representing the errors of measurement.
6. Principle of parsimony
To keep the as simple as possible; representing all other variables.
7. Wrong functional form

Correct variables explaining a phenomenon but not sure about the functional
form. The scattergram: helpful if two-variable model is concerned.

2.6 THE SAMPLE REGRESSION FUNCTION (SRF)

what if we do not have the information on population? What we have is a
sample of Y values corresponding to some fixed Xs.

Table 2.1: presenting the population, not a sample

Table 2.4 AND 2.5: a randomly selected sample of Y values for the fixed Xs.
OUESTION: From the sample of Table 2.4, can we predict the average weekly
consumption expenditure Y in the population as a whole corresponding to the
chosen Xs?
OR
Can we estimate the PRF from the sample data?

TWO RANDOM SAMPLES FROM THE POPULATION GIVEN IN TABLE 2.1:

2.6 THE SAMPLE REGRESSION FUNCTION (SRF) (contd)

Plotting tha data of Tables 2.4 and 2.5:

2.6 THE SAMPLE REGRESSION FUNCTION (SRF) (contd)

the concept of the sample regression function (SRF):

The sample counterpart of (2.2.2) may be written as:

= 1 + 2

(. . )

A particular numerical value obtained by the estimatr in an application is

known as an estimate.

2.6 THE SAMPLE REGRESSION FUNCTION (SRF) (contd)

the concept of the sample regression function (SRF):

The sample counterpart of (2.4.2) may be written as:

= 1 + 2 +

(. . )

(2.6.2): the stochastic form of SRF given in (2.6.1).

: the (sample) residual term; which is an estimate of .

TO SUM UP

Our primary objective in regression analysis is to estimate the PRF

= 1 + 2 +

. .

on the basis of the SRF

= 1 + 2 +

(. . )

WHY? Because more often we do not have data for all the population.

For = , we have one (sample) observation = see tables 2.4 and 2.5 .

In terms of the SRF:

The observed can be expressed as

= +

(. . )

In terms of the PRF:

The observed can be expressed as

= + (. . )

FIGURE 2.5 SAMPLE AND POPULATION REGRESSION LINES

THE CRITICAL QUESTION

Granted that the SRF is but an approximation of the PRF, can we devise a
rule or method that will make this approximation as close as possible?
OR:
How should the SRF be constructed so that 1 is as close as possible to the
true 1 AND 2 is as close as possible to the true 2 even though we will
never know the true 1 2 ?
CHAPTER 3: procedures that tell us how to construct the SRF to mirror the PRF as
faithfully as possible.