I. Silva and M.E.

Silva Parameter estimation for INAR processes based on HOS

Parameter estimation for INAR processes

based on High-Order Statistics

Isabel Silva1 Maria Eduarda Silva2,3

1 Centro das Construes (CEC) e Departamento de Engenharia Civil, Faculdade de Engenharia da Universidade do Porto

2 Grupo de Matemtica e Informtica, Faculdade de Economia da Universidade do Porto

3 Unidade de Investigao Matemtica e Aplicaes (UIMA), Universidade de Aveiro

COMPSTAT 2008

August, 2008 1/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Outline

Introduction
High-Order Statistics (HOS)
INteger-valued AutoRegressive (INAR) processes

Least square estimation using HOS

Monte Carlo results and application to real data
Final remarks

August, 2008 2/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

High-Order Statistics (HOS)

Moments and cumulants of order higher than two

Introduction August, 2008 3/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

High-Order Statistics (HOS)

Moments and cumulants of order higher than two

Lack of Gaussianity and/or non-linearity

Introduction August, 2008 3/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

High-Order Statistics (HOS)

Moments and cumulants of order higher than two

Lack of Gaussianity and/or non-linearity

Notation:
{Xt } : kth-order stationary stochastic process
X (s1 , . . . , sk1 ) : kth-order joint moment of Xt , Xt+s1 . . . , Xt+sk1 , (s1 , . . . , sk1 R)

X (s1 , . . . , sk1 ) = E[Xt Xt+s1 . . . Xt+sk1 ]

X = E[Xt ]

Introduction August, 2008 3/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

INteger-valued AutoRegressive processes

INAR(p) [Latour, 1998]

Xt = 1 Xt1 + 2 Xt2 + + p Xtp + et

Introduction August, 2008 4/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

INteger-valued AutoRegressive processes

INAR(p) [Latour, 1998]

Xt = 1 Xt1 + 2 Xt2 + + p Xtp + et

0 i < 1, i = 1, . . . , p 1, and 0 < p < 1, such that pk=1 k < 1

thinning operation [Steutel and Van Harn, 1979; Gauthier and Latour, 1994)]

i Xti = j=1
ti X
Yi,j , for i = 1, . . . , p,

{Yi,j } (counting series): set of i.i.d. non-negative integer-valued r.v. with

E[Yi,j ] = i , Var[Yi,j ] = i2 and E[Yi,j
3]=
i

{et } (innovation process): sequence of i.i.d. non-negative integer-valued r.v.

(independent of {Yi,j }) with E[et ] = e , Var[et ] = e2 and E[e3t ] = e

Introduction August, 2008 4/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

INteger-valued AutoRegressive processes

INAR(p) [Latour, 1998]

Xt = 1 Xt1 + 2 Xt2 + + p Xtp + et

0 i < 1, i = 1, . . . , p 1, and 0 < p < 1, such that pk=1 k < 1

thinning operation [Steutel and Van Harn, 1979; Gauthier and Latour, 1994)]

i Xti = j=1
ti X
Yi,j , for i = 1, . . . , p,

{Yi,j } (counting series): set of i.i.d. non-negative integer-valued r.v. with

E[Yi,j ] = i , Var[Yi,j ] = i2 and E[Yi,j
3]=
i

{et } (innovation process): sequence of i.i.d. non-negative integer-valued r.v.

(independent of {Yi,j }) with E[et ] = e , Var[et ] = e2 and E[e3t ] = e

Usually: Poisson INAR(p) process with binomial thinning operation

Introduction August, 2008 4/1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Third-order moments of INAR processes

[Silva and Oliveira (2004, 2005) and Silva (2005)]
p p p p p p
X (0, 0) = i j k X (i j, i k) + 3 j i 2 X (i j) + 3X e i 2 + e
i=1 j=1 k=1 i=1 j=1 i=1
p p p p
+ 3X (e2 + e 2 ) i + 3e i j X (i j) + X (i 3i i 2 i3 )
i=1 i=1 j=1 i=1
p
X (0, k) = i X (0, k i) + e X (0), k > 0
i=1
p p p
X (k, k) = i j X (k i, k j) + i 2 X (k i) + 2e X (k) X (e 2 e 2 ), k > 0
i=1 j=1 i=1
p
X (k, m) = i X (k, m i) + e X (k), m>k>0
i=1

Least square estimation using HOS August, 2008 5/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Third-order moments of INAR processes

[Silva and Oliveira (2004, 2005) and Silva (2005)]
p p p p p p
X (0, 0) = i j k X (i j, i k) + 3 j i 2 X (i j) + 3X e i 2 + e
i=1 j=1 k=1 i=1 j=1 i=1
p p p p
+ 3X (e2 + e 2 ) i + 3e i j X (i j) + X (i 3i i 2 i3 )
i=1 i=1 j=1 i=1
p
X (0, k) = i X (0, k i) + e X (0), k > 0
i=1
p p p
X (k, k) = i j X (k i, k j) + i 2 X (k i) + 2e X (k) X (e 2 e 2 ), k > 0
i=1 j=1 i=1
p
X (k, m) = i X (k, m i) + e X (k), m>k>0
i=1

INAR processes have a non-linear structure

st nd
1 and 2 order moments are not sufficient to describe dependence structure
Least square estimation using HOS August, 2008 5/1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Least square estimation using HOS

{x1 , x2 , . . . , xn } : realization of a non-negative integer-valued stationary stochastic
process with third-order moments (0, k), k > 0

Least square estimation using HOS August, 2008 6/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Least square estimation using HOS

{x1 , x2 , . . . , xn } : realization of a non-negative integer-valued stationary stochastic
process with third-order moments (0, k), k > 0
Approximating model: INAR(p) with parameters 1 , , p , e , e2 and
third-order moments X (0, k), k > 0, which can be represented in the following
matrix form:

3,X = M3,X + e X (0)1p

m
X (0, 1) X (0, 0) X (1, 1) X (p 1, p 1) 1

... 1
X (0, 2) X (0, 1) X (0, 0) X (p 2, p 2) 2

... 1
= + (0)

. . . .. . . e X .
. . . . . . .
. . . . . .


X (0, p) X (0, p 1) X (0, p 2) ... X (0, 0) p 1

p p
X (0) = i X (i) + e X + Vp , with Vp = e 2 + X i 2
i=1 i=1

Least square estimation using HOS August, 2008 6/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Least square estimation using HOS

Defining H = [M3,X X (0)1p ] and = [ 1 p e ]T

3,X = H

Least square estimation using HOS August, 2008 7/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Least square estimation using HOS

Defining H = [M3,X X (0)1p ] and = [ 1 p e ]T

3,X = H

may be estimated by least squares, ie, minimizing the squared error between 3,X
and the third-order moments of the data:

3 = [ (0, 1) (0, p) ]T

Least square estimation using HOS August, 2008 7/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Least square estimation using HOS

Defining H = [M3,X X (0)1p ] and = [ 1 p e ]T

3,X = H

may be estimated by least squares, ie, minimizing the squared error between 3,X
and the third-order moments of the data:

3 = [ (0, 1) (0, p) ]T

Least Squares estimator of using HOS (LS_HOS)

= min {L ( )} = min {( 3 H )T ( 3 H )}

Least square estimation using HOS August, 2008 7/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Least square estimation using HOS

Defining H = [M3,X X (0)1p ] and = [ 1 p e ]T

3,X = H

may be estimated by least squares, ie, minimizing the squared error between 3,X
and the third-order moments of the data:

3 = [ (0, 1) (0, p) ]T

Least Squares estimator of using HOS (LS_HOS)

= min {L ( )} = min {( 3 H )T ( 3 H )}

In practice: = min {L ( )} = min {( 3 H )T ( 3 H )}

Least square estimation using HOS August, 2008 7/1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Monte Carlo results

To examine the small sample properties of the LS_HOS
To compare its performance with other estimation methods: YW, CLS and WHT

Monte Carlo results and application to real data August, 2008 8/1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Monte Carlo results

To examine the small sample properties of the LS_HOS
To compare its performance with other estimation methods: YW, CLS and WHT

1000 realizations of Poisson INAR(p) processes, with binomial thinning operation,

for several orders, sample sizes and parameter values

Monte Carlo results and application to real data August, 2008 8/1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Monte Carlo results

To examine the small sample properties of the LS_HOS
To compare its performance with other estimation methods: YW, CLS and WHT

1000 realizations of Poisson INAR(p) processes, with binomial thinning operation,

for several orders, sample sizes and parameter values

Sample properties of the LS_HOS estimator

The sample bias, variance and mean square error decrease as the sample size
increases

Distribution of the estimators is consistent and symmetric

Monte Carlo results and application to real data August, 2008 8/1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Monte Carlo results

To examine the small sample properties of the LS_HOS
To compare its performance with other estimation methods: YW, CLS and WHT

1000 realizations of Poisson INAR(p) processes, with binomial thinning operation,

for several orders, sample sizes and parameter values

Sample properties of the LS_HOS estimator

The sample bias, variance and mean square error decrease as the sample size
increases

Distribution of the estimators is consistent and symmetric

For small sample size: evidence of departure from symmetry in the marginal
distributions, specially for values of the parameter near the non-stationary region

Monte Carlo results and application to real data August, 2008 8/1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Monte Carlo results

N=50 N=200
0.6
0.4
0.2
Bias()

0
0.2
0.4

YW CLS WHT LS_HOS YW CLS WHT LS_HOS

2
Bias()

4
YW CLS WHT LS_HOS YW CLS WHT LS_HOS

Figure: Boxplots of the sample bias for the estimates obtained in 1000 realizations of 50 and 200 observations of the

INAR(1) model: Xt = 0.9 Xt1 + et , where et Po(1)

Monte Carlo results and application to real data August, 2008 9/1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Application to real data

50

45

40

35
Number of plants

30

25

20

15

10

5
1920 1930 1940 1950 1960 1970 1981

Figure: The number of Swedish mechanical paper and pulp mills, from 1921 to 1981 [Brnns (1995) and Brnns and

Hellstrm (2001)]

Monte Carlo results and application to real data August, 2008 10 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the innovation process:

X = 20.40 and S2 = 155.16

Monte Carlo results and application to real data August, 2008 11 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the innovation process:

X = 20.40 and S2 = 155.16

Method e e2 x x2 MSE
CLS 0.9591 0.2017 15.2268 4.9315 192.2764 8.5494
LS_HOS 0.9269 1.3635 19.2253 18.6525 145.4513 7.4465

Table: The parameter estimates of the number of Swedish mechanical paper and pulp mills

Monte Carlo results and application to real data August, 2008 11 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the innovation process:

X = 20.40 and S2 = 155.16

Method e e2 x x2 MSE
CLS 0.9591 0.2017 15.2268 4.9315 192.2764 8.5494
LS_HOS 0.9269 1.3635 19.2253 18.6525 145.4513 7.4465

Table: The parameter estimates of the number of Swedish mechanical paper and pulp mills

e (1 )(e + e2 )
Mean and variance of the estimated models: x = and x2 =
1 (1 )2 (1 + )

Monte Carlo results and application to real data August, 2008 11 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the innovation process:

X = 20.40 and S2 = 155.16

Method e e2 x x2 MSE
CLS 0.9591 0.2017 15.2268 4.9315 192.2764 8.5494
LS_HOS 0.9269 1.3635 19.2253 18.6525 145.4513 7.4465

Table: The parameter estimates of the number of Swedish mechanical paper and pulp mills

e (1 )(e + e2 )
Mean and variance of the estimated models: x = and x2 =
1 (1 )2 (1 + )

MSE between the observations and the fitted models based on LS_HOS and CLS
estimates
Monte Carlo results and application to real data August, 2008 11 / 1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Application to real data

50
Real data
45 CLS
LS_HOS
40

35
Number of plants

30

25

20

15

10

5
1920 1930 1940 1950 1960 1970 1981

Figure: The number of plants and the fitted values considering the LS_HOS and CLS estimates

Monte Carlo results and application to real data August, 2008 12 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Final remarks
Advantage of HOS: capability to detect and characterize the deviations from
Gaussianity and non-linearity of the processes

Final remarks August, 2008 13 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Final remarks
Advantage of HOS: capability to detect and characterize the deviations from
Gaussianity and non-linearity of the processes
INAR processes are non-Gaussian

Parameter estimation method: Least squares using HOS

Minimize the errors between the third-order moment of the observations and of
the fitted model

Final remarks August, 2008 13 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Final remarks
Advantage of HOS: capability to detect and characterize the deviations from
Gaussianity and non-linearity of the processes
INAR processes are non-Gaussian

Parameter estimation method: Least squares using HOS

Minimize the errors between the third-order moment of the observations and of
the fitted model
Monte Carlo results: LS_HOS provides good results, in terms of sample bias,
variance and mean square error

Final remarks August, 2008 13 / 1

 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

Final remarks
Advantage of HOS: capability to detect and characterize the deviations from
Gaussianity and non-linearity of the processes
INAR processes are non-Gaussian

Parameter estimation method: Least squares using HOS

Minimize the errors between the third-order moment of the observations and of
the fitted model
Monte Carlo results: LS_HOS provides good results, in terms of sample bias,
variance and mean square error
When used in the context of a non-Poisson real dataset the LS_HOS estimates
provide a model with mean, variance and autocorrelations closer to the sample
values
Final remarks August, 2008 13 / 1
 I. Silva and M.E. Silva Parameter estimation for INAR processes based on HOS

References
BRNNS, K. (1995).
Explanatory Variables in the AR(1) Count Data Model.
Ume Economic Studies 381.

BRNNS, K. and HELLSTRM, J. (2001).

Generalized Integer-Valued Autoregression.
Econometric Reviews 20 (4), 425-443.

GAUTHIER, G. and LATOUR, A. (1994).

Convergence forte des estimateurs des paramtres dtun processus GENAR(p).
Annales des Sciences Mathmatiques du Qubec 18, 49-71.

LATOUR, A. (1998).
Existence and stochastic structure of a non-negative integer-valued autoregressive process.
Journal of Time Series Analysis 19, 439-455.

SILVA, I. (2005).
Contributions to the analysis of discrete-valued time series.
PhD Thesis. Universidade do Porto, Portugal.

SILVA, M. E. and OLIVEIRA, V. L. (2004).

Difference equations for the higher-order moments and cumulants of the INAR(1) model.
Journal of Time Series Analysis 25, 317-333.

SILVA, M. E. and OLIVEIRA, V. L. (2005).

Difference equations for the higher-order moments and cumulants of the INAR(p) model.
Journal of Time Series Analysis 26, 17-36.

STEUTEL, F. W. and VAN HARN, K. (1979).

Discrete analogues of self-decomposability and stability.
The Annals of Probability 7, 893-899.

References August, 2008 14 / 1