Professional Documents
Culture Documents
MA(q)
White noise processes may not occur very common, but weighted sums of
a white noise process can provide a good representation of processes that
are nonwhite.
The mean of the moving average process is independent of time, since
E (Yt )
The variance:
var(Yt ) 0 E [Yt )2 ]
E [( t 1 t 1 2 t 2 .... q q )( t 1 t 1 2 t 2 .... q q )]
E [ t2 12 t 1 22 t22 .... q2 t2q 1 t 1 t 2 t 2 t ....]
2 12 2 22 2 .... q2 2
2 (1 12 22 .... q2 )
(sin ce E ( t ) 0 for t )
i 1
2
Lets examine some simple moving average processes; calculating the
mean, variance, covariance and autocorrelation function for each. These
statistics are important since:
1. They provide information that helps characterize the process;
2. Help us to identify the process when we construct models.
Example 1
MA(1)
Yt t 1 t 1
mean =
2
2
variance = (1 1 )
covariance for lag one
1 E [(Yt )(Yt 1 )]
E [( t 1 t 1 )( t 1 1 t 2 )]
1 2
For K , K 1, in general
K E [( t 1 t 1 )( t K 1 t K 1 )] 0
Thus, the MA(1) process has a covariance of zero when the displacement
is more than one period. (It has a memory of only one period.)
*Autocorrelation function for MA(1)
1
K
K
1 12
0
0
[Graph MA(1) ]
for K 1
otherwise
3
MA(2)
Example 2
Equation
Yt t 1 t 1 2 t 2
*mean =
2
2
2
*variance = (1 1 2 ) 0
*covariance
1 E [( t 1 t 1 2 t 2 )( t 1 1 t 2 2 t 3 )
1 2 21 2 1(1 2 ) 2
2 E [( t 1 t 1 2 t 2 )( t 2 1 t 3 2 t 4 )
2 2
and
K 0 for K 2.
*Autocorrelation function is
1 1(1 2 )
;
0 1 12 22
2
2
.
0 1 12 22
4
In general, the formula of autocorrelation for a moving average process of
q
order [ MA(q ) ] is
K 1 K 1 .... q K q
1 12 22 .... q2
We can see now why the sample autocorrelation function can be useful in
specifying the order of a moving average process: the autocorrelation
function K for the MA(q ) process has q non-zero values and is zero for
K q.
Invertible
MA(1) process Yt t 1 t 1
If 1 1, the process is invertible, i.e., we can invert the series and
express the current value of Yt in a current disturbance and its lagged value. This
is so called an autoregressive representation.
t Yt 1 t 1
t 1 Yt 1 1 t 2
t 2 Yt 2 1 t 3
M
Yt t 1(Yt 1 1 t 2 )
t 1Yt 1 12 (Yt 2 1 t 3 )
t 1Yt 1 12Yt 2 13 (Yt 3 1 t 4 )
M
t 1Yt 1 12Yt 2 13Yt 3 ...
Notice that the autoregressive representation exists only if 1 1.
1 1 2 ....
AR(1)
Yt 1Yt 1 t
*mean =
and is stationary if 1 1.
1 1
* var iance (set 0)
2
1 12
6
*covariance
1 E [Yt 1(1Yt 1 t )]
1 0 (substitute 0 int o t he equation ) 1
2
1 12
2 E [Yt 2 (1Yt 1 t )]
sin ce Yt 1 Yt 2 t 1, so we have
2 E [Yt 2 (12Yt 2 1 t 1 t )]
2
2
2
(substitute 0 by
2 1
1 12
1 12
Similarly, the covariance for K-lag displacement is
2
1 0
2
K 0
1 12
*the autocorrelation function
K
1
1, K
K
1
K
1K
0
(decline geometrically !)
7
AR (2)
Example 2
Yt 1Yt 1 2Yt 2 t
*mean =
1 1 2
(1)
(2)
1 0 2 1
(3)
1 1 2 0
In general, for K
2, we have
K 1 K 1 2 k 2
2
Now we can solve 0 , 1, and 2 in terms of 1, 2 , and .
Starting from 1 :
1 1 0 2 1 1
1 0
1 2
(4)
0 1 1 12 1 22 0
then substituting (4) into (5)
12 0 212 0
0
22 0 2
1 2 1 2
after rearranging, 0
then solve for 2 .
2
(1 2 )
(1 2 ) [(1 2 ) 12 ]
(5)
8
*autocorrelation function
1
0 1 2
2
12
2
2
0
1 2
In general, for K 2
K 1 K 1 2 K 2
1
1 2
2 2
12
1 2
M
Suppose we have the sample autocorrelation function for a time series
which is AR(2) . Then we calculate the sample autocorrelation function.
)
(Yt Y )(Yt K Y )
T K
t 1
2
(Yt Y )
t 1
1 and 2 .
[Show graph of AR(2) ]
2
T
in magnitude.
1
T
. So 5 percent
10
where
t ~ WN (0, 2 )
Assume that the process is stationary, so that its mean is constant over time:
1
1 2 ... p
1 2 ... p
Notice that this gives a necessary condition for stationarity of Yt
1 2 ... p 1
For variance and covariance, let us assume ARMA(1,1)
Yt 1Yt 1 t 1 t 1
Variance:
0 E[(1Yt 1 t 1 t 1 )(1Yt 1 t 1 t 1 )]
12 0 2 12 2 211 E[ t 1Yt 1 ]
and
E[ t 1Yt 1 ] 2
We have
0 12 0 2 12 2 211 2
or
0
1
* 2 (1 12 211 )
(1 12 )
Covariance
1 E[Yt 1 (1Yt 1 t 1 t 1 )] 1 0 1 2
2 E[Yt 2 (1Yt 1 t 1 t 1 )] 1 1
Autocorrelation
1
(1 11 )(1 1 )
0
1 12 211
For K 2 , K 1 K 1
1
Notice that the autocorrelation function begins at its starting point 1 , which is a
function of 1 and 1 ; then decline geometrically from the starting value. This
reflects the fact that the moving average part of the process has a memory of
only one period.
11
Show graphs
Yt 0.8Yt 1 t 0.9 t 1 2
Yt 0.8Yt 1 t 0.9 t 1 2
p
The BIC estimator , p is the value that minimizes BIC(p), among the possible
choices of p 1, 2,... pmax .
12
AR(p) Model for U.S. Inflation
p
SSR(p)/T ln(SSR(p)/T) (p+1)ln(T)/T
BIC
0
2.853
1.048
0.033
1
2.726
1.003
0.066
2
2.361
0.859
0.099
3
2.264
0.817
0.132
4
2.261
0.816
0.165
5
2.260
0.815
0.198
6
2.257
0.814
0.231
T=152 (1962-1999 qtrly)
1.081
1.069
0.958
0.949
0.981
1.014
1.045
R squares
0
0.045
0.173
0.206
0.207
0.208
0.209
Non-stationary Process
Example:
If Yt is first-order integrated non-stationary series, then
Yt Yt Yt 1
is stationary.
If Yt is second-order integrated series, then
2Yt Yt Yt 1
would be stationary.
If Yt is non-stationary, the statistical characteristics of the process is not
independent of time anymore.
Example: Random walk
Yt Yt 1 t
2
where t ~ WN (0, )
13
0 E (Yt 2 )
E [(Yt 1 t )2 ] E (Yt 21 ) 2
E (Yt 22 ) 2 2 ....
E (Yt 2N ) N 2
The variance is infinite when N approaches infinite.
The same is true for the covariance.
Y
But,
K t is stationary, (white noise), so we have 0 1, K 0for K 0
.
14
t t 1 t
t ~ WN
15
Issues:
1.Regression of a random walk on time t by least squares will produce high R 2
value; i.e., if the true process is 0 , R 2 0.44 just by doing so.
2.If 0 , R 2 will be even higher, and will increase with the sample size and
reach one in the limit.
3.The residuals have on average only about 14% of the true variance.
10
4.The residuals are highly correlated, roughly (1 ) at lag one where T is the
T
sample size.
5.Conventional t-tests are not valid.
6.Regression of one random walk variable on another one is strongly subject to
the spurious regression phenomenon.