Econometrics 2 Fall 2005

Linear Regression with

Time Series Data
Heino Bohn Nielsen

1 of 19

Outline
(1) Interpretation of Time Series Regressions
(2) Assumptions and Results:

(a) Consistency
Example: AR(1)
(b) Unbiasedness and Bias in Dynamic Models
Example: AR(1)
(c) Asymptotic Distribution
(3) Autocorrelation of the Error Term

(a) Consequences of autocorrelation

(b) Interpretations on residual autocorrelation
(c) Tests for no-autocorrelation

2 of 19

Interpretation of Regression Models

Consider the linear regression model:

yt = x0t + t,

t = 1, 2, ..., T.

()

The interpretation depends on the variables included in xt.

If xt contains contemporaneously dated variables it is denoted a static regression.
A simple model for yt given the past is the autoregressive model:

yt = yt1 + t.
Shocks to the process ( t) have dynamic eects.
More complicated dynamics in the autoregressive distributed lag (ADL) model:

yt = 1yt1 + x0t0 + x0t11 + t.

A workhorse in dynamic econometrics.
3 of 19

OLS Estimator
One way to motivate OLS is the so-called moment condition

E[xt t] = 0.
The model implies that

()

= yt x0t , so that
E[xtyt] E[xtx0t] = 0.

If E[xtx0t] is non-singular, that suggests the population estimator

b = E[xtx0 ]1E[xtyt].

In practice we replace the expectations by sample averages. Under a LLN:

T
X
t=1

xtyt E[xtyt] and T

T
X
t=1

xtx0t E[xtx0t].

and we can use the well known OLS estimator

!1
!

T
T
X
X
b = T 1
T 1
xtx0
xtyt .

t=1

t=1

4 of 19

Main Assumption
We impose assumptions to ensure that a LLN applies to the sample averages.

Main Assumption:
Consider a time series yt and the k 1 vector time series xt. We assume
0
(1) that zt = (yt, x0t) has a joint stationary distribution; and
(2) that the process zt is weakly dependent, so that zt and zt+k becomes approximately
independent for k .
Under these assumptions, most of the results for linear regression on random samples
carry over to the time series case.

5 of 19

Consistency
b converges to
A minimal requirement for an estimator is that it is consistent, so that
as we get more and more observations.
Result 1: Consistency
Let yt and xt obey the main assumption. If the regressors are predetermined,

E[xt t] = 0,

(#)

b as T .
then the OLS estimator is consistent, i.e.

Often, it is natural to think of the regression model as the conditional expectation,

E[yt | xt] = x0t.

For this we need the zero-conditional-mean assumption:

E[ t | xt] = 0,

(##)

which is stronger than the no-contemporaneous-correlation (#).

OLS is consistent if the regression represents the conditional expectation of yt | xt.

6 of 19

Consistency of OLS in an AR(1)

Look at an AR(1) model:

yt = yt1 +

t = 1, 2, ..., T .

We write the OLS estimator as

PT
PT
PT
1
1
1
T
T
y
y
(y
+
)y
T
t
t1
t1
t
t1
t=1
t=1
t=1 t yt1
b
=
=

+
,
=
P
P
P
T
T
2
2
2
T 1 t=1 yt1
T 1 t=1 yt1
T 1 Tt=1 yt1

and look at the terms as T :

plim T

plim T 1
T

T
X

t=1
T
X

2
yt1
= q<

t yt1

= E [ tyt1] = 0

(O)
(OO)

t=1

(O) holds for a stationary process (q is the limiting variance of yt).

(OO) follows from the predeterminedness.

7 of 19

Unbiasedness
b, is unbiasedness: E[]
b = .
A stronger requirement for an estimator,

Result 2: Unbiasedness
Let yt and xt obey the main assumption. If the regressors are strictly exogenous,

E [ t | x1, x2, ..., xt, ..., xT ] = 0,

b | x1, x2, ..., xT ] = .

then the OLS estimator is unbiased, i.e. E[

For unbiasedness we need strict exogeneity, which is not fulfilled in a dynamic regression.
Consider the first order autoregressive model

yt = yt1 + t.
Here yt is function of

t,

so

cannot be uncorrelated with yt, yt+1, ..., yT .

Result 3: Estimation bias in dynamic models

In general, the OLS estimator is not unbiased in a dynamic regression model.
8 of 19

Finite Sample Bias in an AR(1)

In a MC simulation we take an AR(1) as the DGP and estimation model:

yt = 0.9 yt1 + t,

N(0, 1).

Mean of OLS estimate in AR(1) model

1.2
1.1
1.0
True value

0.9
0.8

Mean

0.7
0.6
0.5

95% confidence bands

0.4
10

20

30

40

50

60

70

80

90

100
9 of 19

Asymptotic Distribution
To derive the asymptotic distribution we need a CLT; additional restrictions on

t.

Result 4: Asymptotic distribution

Let yt and xt obey the main assumption. Furthermore, assume homoskedasticity and
no serial correlation, i.e.

E[ 2t | xt] = 2
E[ t s | xt, xs] = 0 for all t 6= s.
Then as T , the OLS estimator is asymptotically normal:

b
T N(0, 2E[xtx0t]1).
Inserting natural estimators, we can test hypothesis using

T
!1
X
a
b
.

N ,
xtx0
b2
t

t=1

10 of 19

Complete Dynamic Model

The precise condition for no-serial-correlation looks strange.
Often we (disregard conditioning) and consider whether t and

are uncorrelated.

We say that a model is dynamically complete if

E[yt | xt, yt1, xt1, yt2, xt2, ..., y1, x1] = E[yt | xt] = x0t.
xt contains all relevant information in the available information set.
No-serial-correlation is practically the same as dynamic completeness.
All systematic information in the past of yt and xt is used in the regression model.
This is often taken as an important design criteria for a dynamic regression model.
We should always test for no-autocorrelation in time series models.

11 of 19

Autocorrelation of the Error Term

If Cov( t, s) 6= 0 for some t 6= s, we have autocorrelation of the error term.
This is detected from estimated residual and Cov(bt,bs) 6= 0 is referred to as residual
autocorrelation. Often used synonymously.

Residual autocorrelation does not imply that the DGP has autocorrelated errors.
Autocorrelation is taken as a signal of misspecification. Dierent possibilities:
(I) Autoregressive errors in the DGP.
(II) Dynamic misspecification.
(III) Omitted variables and non-modelled structural shifts.
(IV) Misspecified functional form.
To solution to the problem depends on the interpretation.

12 of 19

Consequences of Autocorrelation
Autocorrelation will not violate the assumptions for Result 1 in general.
But E[xt t] = 0 is violated if the model includes a lagged dependent variable.
Look at an AR(1) model with error autocorrelation, i.e. the two equations

yt = yt1 + t
t = t1 + vt ,
Both yt1 and

depends on

t1,

vt IID(0, 2v ).

so E [yt1 t] 6= 0.

Result 5: Inconsistency of OLS

In a regression model including the lagged dependent variable, the OLS estimator is
not consistent in the presence of autocorrelation of the error term.
Even if OLS is consistent, the standard formula for the variance Result 4 is no longer valid.
It is possible to derive the variance, the so-called heteroskedasticity-and-autocorrelationconsistent (HAC) standard errors.
13 of 19

I: Autoregressive Errors in the DGP

Consider the case where the errors are truly autoregressive:

yt = x0t + t
t = t1 + vt ,

vt IID(0, 2v ).

If is known we can write

(yt yt1) = x0t x0t1 + ( t t1)

yt = yt1 + x0t x0t1 + vt.
The transformation is analog to GLS transformation in the case of heteroskedasticity.
The GLS model is subject to a so-called common factor restriction: three regressors but
only two parameters, and . Estimation is non-linear.

14 of 19

Consistent estimation of the parameters in the GLS model requires

E[(xt xt1) ( t

t1)]

= 0.

But then t1 should be uncorrelated with xt, i.e. that E[ txt+1] = 0.

Consistency of GLS requires stronger assumptions than consistency of OLS.

The GLS transformation is rarely used in modern econometrics.

(1) Residual autocorrelation does not imply that the error term is autoregressive.
There is no a priori reason to believe that the transformation is correct.
(2) The requirement for consistency of GLS is strong.

15 of 19

II: Dynamic Misspecification

Residual autocorrelation indicates that the model is not dynamically complete, so

E[yt | xt] 6= E[yt | xt, yt1, xt1, yt2, xt2, ..., y1, x1].
The (dynamic) model is misspecified and should be reformulated.
Natural remedy is to extend the list of variables in xt.

If autocorrelation seems of order one, then a starting point is the GLS transformation.
But the AR(1) structure is only indicative and we look at the unrestricted ADL model

yt = 0yt1 + x0t1 + x0t12 + t.

Finding of AR(1) is only used to extend the list of regressors. COMFAC removed.

16 of 19

III: Omitted Variables

Omitted variables in general can also produce autocorrelation. Let the DGP be

yt = x1t 1 + x2t 2 + t,

()

and consider the estimation model

yt = x1t 1 + ut.

()

Then the error term is ut = x2t 2 + t, which is autocorrelated if x2t is persistent.

An example is if the DGP exhibit a level shift, e.g. () includes the dummy variable

0 for t < T0
x2t =
.
1 for t T0

If x2t is not included in () the the residual will be systematic.

Again the solution is to extend the list of regressors, xt.

17 of 19

IV: Misspecified Functional Form

If the true relationship

yt = g(xt) +

is non-linear, then the residuals from a linear regression will typically be autocorrelated.

The obvious solution is to try to reformulate the functional form of the regression line.

18 of 19

Test for No-Autocorrelation

Let bt (t = 1, 2, ..., T ) be the residuals from the original regression model

yt = x0t + t.

A test for no-autocorrelation is based on the hypothesis = 0 in the auxiliary regression

bt = x0t + bt1 + ut,

where xt is included because it may be correlated with

t1.

A valid test is tratio for = 0. Alternatively there is the Breusch-Godfrey LM test

LM = T R2 2(1).
Note that xt and

are orthogonal, and any explanatory power is due to bt1.

The Durbin Watson (DW) test is derived for finite samples.

Based on strict exogeneity. Not valid in many models.

19 of 19