Autoregressive-Moving Average (ARMA) Models

Lesson 9: Autoregressive-Moving
Average (ARMA) models
Umberto Triacca
Dipartimento di Ingegneria e Scienze dellInformazione e Matematica
Universita dellAquila,
umberto.triacca@ec.univaq.it
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Introduction
We have seen that in the class of stationary, zero mean,

Gaussian processes the probabilistic structure of a stochastic
process is completly characterized by the autocovariance
function.

Autocovariance function
Stationary, zero mean, Gaussian process
'$
DGP x (k)
&%
7

?
x1 , ..., xT

Introduction
However, in general, to know the autocovariance function

means to know a sequence composed by an infinite number of
elements.
We have to estimate a infinite number of parameters
x (0), x (1), x (2), ...,
from observed data.
This mission is impossible

Introduction
We introduce a very important class of stochastic processes,

which autocovariance functions depend on a finite number of
unknown parameters:
the class of the AutoregRessive Moving Average (ARMA)

processes.

Autoregressive-Moving Average (ARMA) models
Definition. The process {xt ; t Z} is an autoregressive
moving average process of order (p, q), denoted with
xt ARMA(p, q),
if
xt 1 xt1 ... p xtp = ut + 1 ut1 + ... + q utq t Z,
where ut WN(0, u2 ), and 1 , ..., p , 1 , ..., q are p + q

constants and the polynomials
(z) = 1 1 z ... p z p
and
(z) = 1 + 1 z... + q z q
have no common factors.
For q = 0 the process reduces to an autoregressive process of

order p, denoted with xt AR(p),
xt 1 xt1 ... p xtp = ut t Z,
For p = 0 to a moving average process of order q, denoted

with xt MA(q)
xt = ut + 1 ut1 + ... + q utq t Z,

An example of Autoregressive-Moving Average
(ARMA) process
The process {xt ; t Z} defined by
xt = 0.3xt1 + ut + 0.7ut1 t Z,
where ut WN(0, u2 ), is an ARMA(1,1) process.

Here
(z) = 1 0.3z
and
(z) = 1 + 0.7z.

An example of Autoregressive-Moving Average
(ARMA) process
A realizzation of the ARMA(1,1) process

xt = 0.3xt1 + ut + 0.7ut1 is presented in the following figure.

An example of Autoregressive (AR) process
xt = 0.7xt1 0.5xt1 + ut t Z,
where ut WN(0, u2 ), is an AR(2) process.

Here
(z) = 1 0.7z + 0.5z 2

An example of Autoregressive (AR) process
A realizzation of the AR(2) process

xt = 0.7xt1 0.5xt2 + ut is presented in the following figure.

An example of Moving Average (MA) process
xt = ut + 0.7ut1 t Z,
where ut WN(0, u2 ), is an MA(1) process.

Here
(z) = 1 + 0.7z

An example of Moving Average (MA) process
A realizzation of the MA(1) process xt = ut + 0.7ut1 is

presented in the following figure.

An example of over-parameterization
Consider the process {xt ; t Z} defined by
xt = xt1 0.21xt2 + ut 0.7ut1 t Z,
where ut WN(0, u2 ).
This process looks like an ARMA(2,1) process but it is not an

ARMA(2,1) process.

An example of over-parameterization
Here
(z) = 1 z + 0.21z 2 = (1 0.7z)(1 0.3z)
and
(z) = 1 0.7z
We note that both polynomials have a common factor, namely
1 0.7z. Discarding the common factor in each leaves
(z) = 1 0.3z
and
(z) = 1.
Thus the process is an AR(1) process, defined by
xt = 0.3xt1 + ut
Causal Autoregressive-Moving Average (ARMA)
models
Definition. An ARMA(p, q) process {xt ; t Z} is causal

(strictly, a causal function of {ut ; t Z}) if there exists
constants 0 , 1 , ... such that

X
|j | <
j=0
and
X
xt = j utj t.
j=0

Here, it is important to clarify the meaning of equality

X
xt = j utj t
j=0
It means that
!2
n
X
lim E xt j utj = 0.
n
j=0
The equality is defined in terms of a limit in the quadratic

mean.

The following two theorems provide, respectively, a

characterization of the of causality and stationarity of an
ARMA(p, q) process.

Theorem. An ARMA(p, q) process {xt ; t Z} is causal if

and only if
(z) = 1 1 z ... p z p 6= 0 for all |z| 1.
Theorem. An ARMA(p, q) process {xt ; t Z} is stationary

if and only if
(z) = 1 1 z ... p z p 6= 0 for all |z| = 1.

The causality and the stationarity of an ARMA process depend

entirely on the autoregressive parameters and not on the
moving-average ones.

Further, we note that if an ARMA(p, q) process is causal, then

is stationary, but stationarity does not imply causality.

Consider, for example, the following AR(1) process:
xt = 3xt1 + ut
where ut WN(0, u2 ). We have that
(z) = 1 3z 6= 0 for all |z| = 1.
and hence the process is stationary, but non causal since
(z) = 1 3z = 0 for z = 1/3.

An important result: There is a one-to-one correspondence
between the parameters of a causal ARMA(p,q) process and
the autocovariance function.

It is important to underline that if we consider the set of

autocorrelation functions there is not a one-to-one
correspondence between the parameters of a causal
ARMA(p,q) process and the autocorrelation function.

Consider the following two MA(1) processes.
xt = ut + ut1
where ut WN(0, u2 ), with || < 1 and

1
yt = ut + ut1

where ut WN(0, u2 ).
Since
1/
= ,
1+ 2
1 + (1/)2
we have that both processes share the same autocorrelation
function. Thus it cannot be used to distinguish between the
two parametrizations.
This example shows that an MA(1)-process is not uniquely

determined by its autocorrelation function. There is an
identification problem with the MA(1) models.
In general, (if all roots of (z) = 0 are real) there can be 2q

different MA(q) processes with the same autocorrelation
function.

Definition. An ARMA(p, q) process {xt ; t Z} is invertible

(strictly, an invertible function of {ut ; t Z}) if there exists
constants 0 , 1 , ... such that

X
|j | <
j=0
and
X
ut = j xtj t.
j=0

The following theorem provides a necessary and sufficient

condition for the invertibility.
Theorem. An ARMA(p, q) process {xt ; t Z} is invertible

if and only if
(z) = 1 + 1 z + ... + q z q 6= 0 for all |z| 1.

We note that an AR(p) process is always invertible, even if it

is non-stationary, while an MA(q) process is always stationary,
even if it is non-invertible.

The invertibility can be used in order to ensure the

identifiability of MA processes.

In general, (if all roots of (z) = 0 are real) there can be 2q
different MA(q) processes with the same autocorrelation
function, but only one of these is invertible.

Conclusion
In the class of the mean-zero causal and invertible Gaussian
ARMA processes there is a one-to-one correspondence
between the family of the finite dimensional distributions of
the process and the finite parametric representation of process.

Conclusion
In the class of the mean-zero causal and invertible

Gaussian ARMA processes the probabilistic properties of the
process are completely characterized by the finite set of
parameters
1 , 2 , ..., p , 1 , 2 , ..., q , u2

Now, we have to estimate a finite number (p + q + 1) of

parameters from observed data.
This mission is possible.

Conclusion
Zero-mean causal invertible Gaussian ARMA process
'$
DGP {1 , 2 , ..., p , 1 , 2 , ..., q , u2 }

&%
7

?
x1 , ..., xT


Autoregressive-Moving Average (ARMA) Models

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Autoregressive-Moving Average (ARMA) Models

Uploaded by

Copyright:

Available Formats

Lesson 9: Autoregressive-Moving

Average (ARMA) models

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

We have seen that in the class of stationary, zero mean,

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Stationary, zero mean, Gaussian process

However, in general, to know the autocovariance function

We have to estimate a infinite number of parameters

x (0), x (1), x (2), ...,

from observed data.

This mission is impossible

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

We introduce a very important class of stochastic processes,

the class of the AutoregRessive Moving Average (ARMA)

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

xt 1 xt1 ... p xtp = ut + 1 ut1 + ... + q utq t Z,

where ut WN(0, u2 ), and 1 , ..., p , 1 , ..., q are p + q

For q = 0 the process reduces to an autoregressive process of

xt 1 xt1 ... p xtp = ut t Z,

For p = 0 to a moving average process of order q, denoted

xt = ut + 1 ut1 + ... + q utq t Z,

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

The process {xt ; t Z} defined by

where ut WN(0, u2 ), is an ARMA(1,1) process.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

A realizzation of the ARMA(1,1) process

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

The process {xt ; t Z} defined by

where ut WN(0, u2 ), is an AR(2) process.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

A realizzation of the AR(2) process

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

The process {xt ; t Z} defined by

where ut WN(0, u2 ), is an MA(1) process.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

A realizzation of the MA(1) process xt = ut + 0.7ut1 is

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Consider the process {xt ; t Z} defined by

xt = xt1 0.21xt2 + ut 0.7ut1 t Z,

This process looks like an ARMA(2,1) process but it is not an

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

(z) = 1 z + 0.21z 2 = (1 0.7z)(1 0.3z)

Definition. An ARMA(p, q) process {xt ; t Z} is causal

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Here, it is important to clarify the meaning of equality

The equality is defined in terms of a limit in the quadratic

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

The following two theorems provide, respectively, a

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Theorem. An ARMA(p, q) process {xt ; t Z} is causal if

(z) = 1 1 z ... p z p 6= 0 for all |z| 1.

Theorem. An ARMA(p, q) process {xt ; t Z} is stationary

(z) = 1 1 z ... p z p 6= 0 for all |z| = 1.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

The causality and the stationarity of an ARMA process depend

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Further, we note that if an ARMA(p, q) process is causal, then

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Consider, for example, the following AR(1) process:

where ut WN(0, u2 ). We have that

(z) = 1 3z 6= 0 for all |z| = 1.

and hence the process is stationary, but non causal since

(z) = 1 3z = 0 for z = 1/3.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models