ch02 1

Economics A Morden Approach -
Prof. Wang Li
1
The Simple Regression
Model (1)
y = |
0
+ |
1
x + u
Prof. Wang Li
2
Chapter Outline
Definition of the Simple Regression Model
Deriving the Ordinary Least Squares
Estimates
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of the OLS
estimators
Regression through the Origin
Prof. Wang Li
3
Lecture Outline
Simple linear regression model
Some terminology
A simple assumption
Zero Conditional Mean Assumption
Ordinary Least Squares
Deriving OLS Estimates
A brief introduction to EViews
Prof. Wang Li
4
Equation y = |
0
+ |
1
x + u
has only one nonconstant regressor x,
it is called a simple linear regression
model,
or two-variables regression model,
or bivariate linear regression model.
y = |
1
x if u = 0
Thus, the change in y is simply |
1
multiplied
by the change in x.
Simple linear regression model
Prof. Wang Li
5
Some Terminology
In the linear regression model,
where y = |
0
+ |
1
x + u, we typically refer to
left-hand side variable y as the
Dependent Variable, or
Explained Variable, or
Response Variable, or
Predicted Variable, or
Regressand
Prof. Wang Li
6
In the simple linear regression of y on x,
we typically refer to right-hand side
variable x as the
Independent Variable, or
Explanatory Variable, or
Control Variable , or
Predictor Variable , or
Regressor
Some Terminology, cont.
Prof. Wang Li
7
The coefficients |
0
, |
1
are called the
regression coefficients.
|
0
is also called the constant term or the
intercept term, or intercept parameter.
|
1
represents the marginal effects of the
regressor, x. It is also called the slope
parameter.
Prof. Wang Li
8
The variable u is called the error term
or disturbance in the relationship.
It represents factors other than x that
can affect y.

u education Earnings + + =
1 0
| |
Prof. Wang Li
9
Meaning of linear:
linear means linear in parameters, not
necessarily mean that y and x must have a
linear relationship.
There are many cases that y and x have
nonlinear relationship, but after some
transformation, they are linear in
parameters.
For example, y=e
|
0
+|
1
x+u

natural log: ln(y)=|
0
+|
1
x+u
Meaning of linear
Prof. Wang Li
10
Example of
Returns to Education
u education wage + + =
1 0
| |
A model of human capital investment
implies getting more education should
lead to higher earnings

education , wage and u are random
variables in the equation.

Prof. Wang Li
11
A Simple Assumption
The average value of u, the error term, in the
population is 0. That is,
expected value of u E(u) = 0
It it restrictive?
This is not a restrictive assumption, since we
can always use |
0
to normalize E(u) to 0.
If for example, E(u)=5. Then
y = (|
0
+5)+ |
1
x + (u-5),
therefore, E(u)=E(u-5)=0.
As long as the intercept |
0
is included in the
equation, the assumption of E(u) = 0 holds.
Prof. Wang Li
12
We need to make a crucial assumption about
how u and x are related
This assumption involves the expected
conditional distribution value of u given any
value of x.
E(u|x) = E(u)
It means that, for any given value of x, the
average of the unobservables is the same,
that u and x are completely unrelated.
therefore E(u|x) must equal the average
value of u in the entire population.
Prof. Wang Li
13
What does it mean?
u and x are completely unrelated
In the example of education, suppose u
represents innate ability, conditional mean
assumption E(u|x) = E(u) means
E(ability|edu=6)=E(ability|edu=18)
The average level of ability is the same
regardless of years of education.
Since we have assumed E(u) = 0, therefore,
E(u|x) = E(u) = 0
Prof. Wang Li
14
Use x to predict the average value of y
take expectations conditional on x
E[y|x] = E[|
0
+ |
1
x+u|x]
= |
0
+ |
1
x+ E[u|x]
Zero Conditional mean assumption:
E(u|x) = E(u) = 0
We got, E[y|x] = |
0
+ |
1
x
Its important implication:
The population regression function is a linear
function of x, where for any x the
distribution of y is centered about E(y|x).
Prof. Wang Li
15
.
.
x
1
=5 x
2
=10
E(y|x) = |
0
+|
1
x
y
f(y)
Conditional
distribution of y
given x
The population
regression function
(PRF)
Prof. Wang Li
16
Deriving the Ordinary Least
Squares Estimates
Basic idea of regression is to estimate the
population parameters from a sample set.
Let {(x
i
,y
i
): i=1, ,n} denote a random
sample of size n from the population.
Since these data come from simple
regression model, for each observation in
this sample, it will be the case that
y
i
= |
0
+ |
1
x
i
+ u
i
Prof. Wang Li
17
.
.
.
.
y
4
y
1
y
2
y
3
x
1
x
2
x
3
x
4
}
}
{
{
u
1
u
2
u
3
u
4
x
y
Population regression line, sample data points
and the associated error terms
PRF
E(y|x) = |
0
+ |
1
x
Prof. Wang Li
18
We must decide how to use these data to
obtain estimates of the intercept and slope
in the population regression
The method of Ordinary Least Squares (OLS)
plays a major role to obtain estimators.
OLS estimators are the best linear unbiased
estimators.
The name ordinary least squares comes
from the fact that these estimates minimize
the sum of squared residuals.
Squares Estimates
Prof. Wang Li
19
To derive the OLS estimator we need to
realize that our main assumption of
E(u|x) = E(u) = 0 also implies that
u and x are uncorrelated, independent
Covariance Cov(x,u) = E(xu) = 0
Why? Remember from basic probability that
Cov(X,Y) = E(XY) E(X)E(Y) and
E(XY)= E(X)E(Y) when x and y are independent.
E(u|x) = E(u)=0 Cov(x,u) = E(xu) = 0
Squares Estimates
Prof. Wang Li
20
Equations E(u) = 0 and E(xu) = 0 can be
written just in terms of x, y, |
0
and |
1
:
since u = y |
0
|
1
x
Zero mean: E(y |
0
|
1
x) = 0
Zero conditional mean: E[x(y |
0
|
1
x)] = 0
This two Equations imply 2 restrictions on
the joint probability distribution of (x,y) in
the population.
Technically, values of the independent
variable is the same as treating the x
i
as
fixed in repeated samples.
Deriving OLS continued
Prof. Wang Li
21
The method of moments approach to
estimation implies imposing the population
moment restrictions on the sample moments
What does this mean?
Recall that for E(X), the mean of a population
distribution, a sample estimator of E(X) is
simply the arithmetic mean of the sample
These are called moment restrictions.
Deriving OLS using M.O.M
Prof. Wang Li
22
We want to choose values of the parameters that
will ensure that the sample versions of our
moment restrictions are true
E(y |
0
|
1
x) = 0
E[x(y |
0
|
1
x)] = 0
The sample versions of our 2 restrictions above
are as follows:
( )
( )
=
=
0

0

1
1 0
1
1
1 0
1
n
i
i i i
n
i
i i
x y x n
x y n
| |
| |
Deriving OLS using M.O.M
Then we are going to solve the equation set.
Prof. Wang Li
23
More Derivation of OLS
Given the definition of a sample mean, and
properties of summation, we can rewrite
the 2 restrictions as follows:
x y
x y
1 0
1 0

or
,

| |
| |
=
+ =
where y is the sample average
of the y
i
24
More Derivation of OLS
( ) ( )
( ) ( )
( )( ) ( )

= =
= =
=
=
=
=
n
i
i i
n
i
i
n
i
i i
n
i
i i
n
i
i i i
x x y y x x
x x x y y x
x x y y x
1
2
1
1
1
1
1
1
1 1
0

|
|
| |
The sample versions of the second restriction
( ) 0

1
1 0
1
=
n
i
i i i
x y x n | |
From basic properties of the summation operator

( ) ( )( ) y y x x y y x
i
n
i
i
n
i
i i
=

= = 1 1

x y
1 0

| | =
Prof. Wang Li
25
the OLS estimated slope
( )( )
( )
( ) 0 that provided
1
2
1
2
1
1
>
=
=
=
n
i
i
n
i
i
n
i
i i
x x
x x
y y x x
|
So the OLS estimated slope is
the numerator is
sample covariance
between x and y
The denominator is the
sample variance of x
Dividing both the numerator and the
denominator by n-1 will make sense.
Prof. Wang Li
26
Summary of OLS slope estimate
The slope estimate is the sample
covariance between x and y divided by the
sample variance of x.
If x and y are positively correlated, the
slope will be positive.
If x and y are negatively correlated, the
slope will be negative.
Only need x to vary in our sample.
Prof. Wang Li
27
the OLS estimated intercept
( )( )
( )
=
=

=
n
i
i
n
i
i i
x x
y y x x
1
2
1
1
|
Once we have the slope estimate:
x y
1 0

| | =
It is straightforward to obtain the
intercept estimate, given y and x.
Prof. Wang Li
28
More OLS
Intuitively, OLS is fitting a line through
the sample points.
The residual, , is an estimate of the
error term, u, and is the difference
between the fitted line (sample regression
function) and the sample point.

i i i
u x y

1 0
+ + = | |
i i
x y
1 0

| | + =
Sample regression function
Prof. Wang Li
29
.
.
.
.
y
4
y
1
y
2
y
3
x
1
x
2
x
3
x
4
}
}
{
{
4
x
y
Sample regression line, sample data points
and the associated estimated error terms
(residuals)
x y
1 0

| | + =
OSL regression line
The notation y emphasizes
that the predicted values
from this equation
are estimates
Prof. Wang Li
30
Alternate approach
The residual for observation i is the
difference between the actual y
i
and its
fitted value:
( ) ( )

= =
= =
n
i
i i
n
i
i
x y u L
1
2
1 0
1
2

| |
Given the intuitive idea of fitting a line, we
can set up a formal minimization problem.
That is, we want to choose our parameters
minimizing the following:
i i i i i
x y y y u
1 0

| | = =
Prof. Wang Li
31
Alternate approach, continued
We use calculus to solve the minimization
problem for the two parameters , The
first order conditions:
( )
( ) 0

2
0

2
1
1 0
1
1
1 0
0
= =
c
c
= =
c
c
=
=
n
i
i i i
n
i
i i
x y x
L
x y
L
| |
|
| |
|
That is the following:
( ) ( )

= =
=
n
1 i
2
i 1 0 i
n
1 i
2
i

x y u
1 0

min
)
(
Prof. Wang Li
32
Alternate approach, continued
These are the same as we have before
(sample versions of 2 restrictions, moment
restrictions ), multiplied by -2n.
( )
( )
=
=
=
=
0

0

1
1 0
1
1 0
n
i
i i i
n
i
i i
x y x
x y
| |
| |
So we call these are The first order
conditions restrictions .
Prof. Wang Li
33
{(x
i
,y
i
): i=1, ,n} are actual values in a
random sample set from the population.
Given a random sample set, the OLS method
is used to estimate the slope and intercept
parameters in the population model.
In population regression function
E(y|x) = |
0
+ |
1
x , the |
0
and |
1
are true values.
Summary of OLS estimate
In the OSL regression line
are estimated values by given random
sample set. are predicted values forming
the fitted line.
1 0

,
| |
x y
1 0

| | + =
y
Which of the followings are right?

(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
t t
X Y | o + =
t t t
X Y | o + + =
(2)

(6)
(7)
t t t
X Y | o + + =

t t t
X Y | o + + =

t t
X Y | o

+ =
t t
X Y | o

+ =
t t t
X Y | o

+ + =
t t t
X Y | o

+ + =
n t 2 , 1 =
Checking Understanding
Prof. Wang Li
35
Lecture Summary
Introduce the simple linear regression
model.
Introduce the method of Ordinary Least
Squares to estimate the slope and
intercept parameters using data from a
random sample.
The name ordinary least squares comes
from the fact that these estimates
minimize the sum of squared residuals.
Prof. Wang Li
36
A brief introduction
to EViews
Using Eviews for OLS regression
Prof. Wang Li
37
EViews Workfile
EViews provides sophisticated data analysis,
regression, and forecasting tools on
Windows-based computers.
A workfile is simply a container for EViews
objects
Most of your work in EViews will involve
objects such as data series, groups,
equations, and vectors, that are contained
in a workfile
Your first step in any project will be to
create a new workfile.
Prof. Wang Li
38
EViews Workfile Window
Prof. Wang Li
39
EViews Workfile
Creating a Workfile by Describing its
Structure
Select File/New Workfile... from the main
menu to open the Workfile Create dialog.
You will select (depending on your Dataset):
Unstructured for Cross-Section
Dated regular frequency for time series
Balanced Panel for a simple panel dataset
Prof. Wang Li
40
EViews Workfile
The default coefficient vector
EViews stores the estimated
coefficients in it.
A default series to hold the
estimated residuals
Prof. Wang Li
41
Getting Data from Excel
Do your own project
You create a workfile by describing its
structure
you may put data into Excel file before
You need import data from .xls
EViews objects also have procedures, or
procs
Click on Procs/Import/Read Text-Lotus-Excel
Prof. Wang Li
42
Getting Data from Excel
Prof. Wang Li
43
Creating a Workfile from a
Foreign Data Source
You may directly open a foreign data
source as an EViews workfile.
Foreign Data are the non-EViews format
data
In the second approach, you simply open
and read data from a foreign data source.
EViews will analyze the data source, create
a workfile, and then automatically import
your data.
Prof. Wang Li
44
Foreign Data Source
Import your data by Copy and Paste
EViews will open the foreign data source
and to read the data into an new EViews
workfile.
Copy the foreign data source to the Windows
clipboard
Right click on the gray area in your EViews
window.
Select Paste as new Workfile
Such an approach, while convenient, is only
practical for small amounts of data.
Prof. Wang Li
45
Foreign Data Source
You may directly open a foreign data
source as an EViews workfile.
First select File/Open/Foreign Data as
Workfile..., to bring up the file Open
dialog.
Clicking on the Files of type combo box
brings up a list of the file types that
EViews currently supports for opening a
workfile.
Prof. Wang Li
46
Stata file
We need to import Stata files to
illustrate examples in text book and do
computer excercises.
File/Open/Foreign Data
as Workfile
the Files of type
combo box
Prof. Wang Li
47
Creating a Workfile from
a Stata file
EViews will open
the selected file,
validate its type,
and display a
tabbed dialog.
Select variables
tab should be
used to choose
the series data
to be included.
Prof. Wang Li
48
Creating a Workfile from
a Stata file
EViews reads
the specific
data into a
new workfile.
It opens the
imported data
as a group
window in a
spreadsheet
view.
Prof. Wang Li
49
Make Equation
Create an equation object
select Object/New Object.../Equation
Opens Equation Specification dialog box
Specifying an equation by List
wage c educ
Here c is a built-in EViews series that is
used to specify a constant in a regression.
u educ wage + + =
1 0
| |
Prof. Wang Li
50
Make Equation
You may include auto-series in the list of
variables. If the auto-series expressions
contain spaces, they should be enclosed in
parentheses.
For example: modify the equation
log(wage) c educ
That is
u educ wage + + =
1 0
) log( | |
Press the Name button and name the equation
as edu1
The edu1 equation object will be placed in
the workfile
Prof. Wang Li
51
Estimating a Regression Model
Perform your estimation by the sample
Press the Estimate button
Choose Estimation method --LS least
squares
and choose sample range
Then click the OK () button
Output view containing estimation results
appears
For any object you can view it in many ways.
An equation also has a representation view
and so on.
Prof. Wang Li
52
Forecasting
Forecasting from an Estimated Equation
You may perform forecasts based upon this
model with the actual data.
Click on the Forecast button in the equation
to open the forecast dialog
Static Forecast perform a series of one-
step ahead forecasts
Click the OK button
EViews will present a graph of the forecast
of the model
the fitted line along with the 95%
confidence intervals for this forecast.
Prof. Wang Li
53
Forecasting
Sample range
By default, EViews will fill the forecast
series with the values of the actual
dependent variable (the workfile sample)
If you want to do out-of-sample forecast
Procs/Resize Current page
Insert data for out-of-sample
independent variables before performing
forecast
Prof. Wang Li
54
Forecasting
Note that you are responsible for supplying
the values for the independent variables in
the out-of-sample forecasting period
After putting data for out-of-sample
independent variables, You must specify the
sample range to be used for the forecast.
You can find forecast results of
dependent variables in forecast series,
for example wagef

ch02 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ch02 1

Uploaded by

Copyright:

Available Formats

Economics A Morden Approach -

Which of the followings are right?

You might also like