11: Some Nonlinear Time-Series Models

SOME NONLINEAR TIME-SERIES MODELS

Himadri Ghosh
Indian Agricultural Statistics Research Institute, New Delhi-11012

1. Introduction:
Well-known Box-Jenkins Autoregressive integrated moving average (ARIMA) methodology has
virtually dominated analysis of time-series data since 1930s. However, it is applicable to only
those data that are either stationary or can be made so. Another limitation is that the resultant
model is Linear. During the last two decades or so, the area of Nonlinear time-series is
rapidly growing. Here, there are basically two possibilities, viz. Parametric or Nonparametric
approaches. Evidently, if in a particular situation, we are quite sure about the functional form,
we should use the former, otherwise the latter may be employed.
Although several such parametric families have been developed, the three most widely used are
discussed below in Section 2. Finally, Section 3 deals with some important Nonparametric
time-series models.
2. Parametric time-series models:
(i) Bilinear time-series model:
Perhaps the most natural way to introduce nonlinearity into a linear ARIMA model is to add
product terms. By restricting to products of time-series variable Xt-j and errors t-k , we end up
with a bilinear model BL(p,q,r,s), introduced by Granger and Anderson (1978) :
p

j =1

k =1

Xt = a j X t j + t + bk t k + c jk X t j t k
j =1 k =1

(1)

where t IID(0,2), ai , bk , and cjk are parameters. Here p denotes lag in the linear part of
autoregression in Xt, while r and s are lags in the nonlinear terms involving past observations and
error series {t}. The coefficients in past observations depend on past shocks thus enabling the
model to capture data with high level crossings.
Although bilinear models were proposed almost two decades ago and a number of
theoretical contributions were made in succeeding period, yet application of such models to real
data is still a challenging task. Recently, Ghosh et al. (2005, 2006b), have developed computer
programs in C-language for fitting of bilinear models. As mentioned earlier, bilinear models are
of particular importance to describe those data sets that depict sudden bursts of large amplitude
at irregular time epochs. As an illustration, Indias marine products export data during the period
1961-62 to 1998-99 is considered. Based on normalized Akaike information criterion (NAIC),
appropriate bilinear time-series model is fitted by applying Newton-Raphson iterative procedure.

124

11: Some Nonlinear Time-Series Models

(ii) Autoregressive conditional heteroscedastic time-series models:

Another important parametric nonlinear time-series family is that of Autoregressive
Conditional Heteroscedastic (ARCH) models. These were developed by R. F. Engle in 1982 to
model varying (conditional variance) or volatility of a time-series. It is often found that larger
values of time-series also lead to larger instability (i.e. larger variances), which is termed
conditional heteroscedasticity. An ARCH model is defined as
= t t , t2 = 0 + 1 Xt21 + ... + q Xt2q ;
where t IID(0,1) and i 0 are parameters.
Xt

t = 1, 2, ..., n

(2)

EViews software package may be employed for fitting of these models. Ghosh and
Prajneshu (2003) applied these models to monthly onion price data during April, 1996 to
October, 2001. Volatility in the data is clearly reflected as the onion price jumped to the
maximum of Rs. 1000/- during October, 1998 to December, 1998. Subsequently, it decreased to
the range of Rs. 125/- to Rs. 500/- during the later time period till October, 2001. As the
assumption of constant one-period ahead forecast variance did not hold, ARCH process was
fitted and out-of-sample forecasts for four months were developed.
(iii) Mixture time-series models:
Mixture nonlinear time-series models may be employed to describe those data sets that
depict sudden bursts, outliers and flat stretches at irregular time epochs. Various models, viz.
Gaussian Mixture Transition (GMTD), Mixed Autoregressive (MAR) and MAR- Autoregressive
Conditional Heteroscedastic (MAR-ARCH) are thoroughly studied by Ghosh et al. (2005,
2006a). Weekly wholesale onion price data during April, 1998 to November, 2001 is considered.
After eliminating trend, seasonal fluctuations are studied by fitting Box-Jenkins airline model to
residual series. The tests for presence of nonseasonal and seasonal stochastic trends and use of
appropriate filters in airline models are also examined. Presence of ARCH is tested by Lagrange
multiplier (LM) test. Estimation of parameters is done using Expectation Maximization (EM)
algorithm and the best model is selected on basis of Bayesian Information Criterion (BIC). Outof-sample forecasting is performed for one-step and two-step ahead prediction by naive
approach. It is concluded that, for data under consideration, a three-component MAR and a twocomponent MAR-ARCH is the best in respective classes. Further, identified MAR-ARCH model
is also shown to perform better than three-component MAR model identified earlier in terms of
having fewer numbers of parameters and lower BIC value.
(iv) Threshold autoregressive time-series models:
The third important parametric nonlinear time-series family is that of Threshold
Autoregressive (TAR) models, initiated by H. Tong in 1980. These assume different linear forms
in different regions of the state space. The division of the state-space is usually dictated by one
threshold variable, say Xt - d, for some d 1. The model is of the form
X t = a0( i ) + ai( i ) X t 1 + ... + a (pi ) X t p + t( i ) , if Xt-d Ri

125

(4)

11: Some Nonlinear Time-Series Models

for i = 1,,k, where Ri forms a (nonoverlapping) partition of the real line and

t( i )

IID(0, i2 ). The simplest thresholding model is the two-regime (i.e. k = 2) TAR model with R1 =

{Xt-d r}. The AR parameter depends on whether Xt-d exceeds the threshold value r; hence the
name TAR. Further, if r = , TAR model reduces to AR(1) model. A comprehensive
discussion of TAR models is given in Tong (1995). STAR software package may be used for
fitting of these models. A heartening feature of TAR models is that these are capable of
describing cyclical fluctuations. Ghosh et al. (2005, 2006c) applied this model to describe
countrys lac export data during the period 1901- 2001. The data set exhibited prominent cycles
of approximately 13 years. The best-identified model on the basis of minimum NAIC value is
found to be two-regime TAR model with delay parameter d =1 and p = 2. Finally, out-of-sample
forecasts for years 2002 and 2003 were obtained.
3. Nonparametric time-series analysis: Parametric families discussed above provide powerful
tools for analyzing time-series data when models are correctly specified. However, any
parametric model is at best only an approximation to the true stochastic dynamics that generates
a given data set. A time-series may be of the type for which there is no suitable parametric model
that gives a good fit to the data under consideration. In such a situation, a newly developing
Nonparametric approach may be employed (Fan and Yao, 2003).
(i) Functional-coefficient autoregressive model:
A very versatile model of the above type is Functional-coefficient autoregressive (FAR)
model introduced by R. Chen and R.S. Tsay in 1993. this model is of the form
(7)
X t = a1 ( X t d ) X1 + ... + ap ( X t d ) X p + ( X t d ) t
where t IID(0,1) and is independent of Xt-1, Xt2, The FAR model depends critically on
choice of model-dependent variable Xt-d, which is one of the lagged variables. A generalization
of this class of models is to allow a linear combination of past values as a model-dependent
variable. Accordingly, the whole class of Adaptive FAR (AFAR) models has been developed
(Fan and Yao, 2003). Recently, Ghosh et al. (2010) applied FAR model for forecasting of Indias
annual export lac data during the period 1900 to 2000.
(ii) Wavelet analysis:
Currently, there is a lot of interest in employing Wavelet analysis for nonparametric
nonlinear time-series modelling. Novel idea of wavelets is that these are localized in both time
and space whereas traditional Fourier bases are localized only in frequency but not in time. The
theory of wavelets permits decomposition of functions into localized oscillating components and
so is an ideal tool for modelling and forecasting purposes. An excellent description of various
aspects of wavelet methodology is given in Abramovich et al. (2000). A further improvement in
this methodology is incorporation of concept of thresholding. Various types of thresholding
are discussed in Percivel and Walden (2000) and Jansen (2001).

126

11: Some Nonlinear Time-Series Models

Illustrations
Problem. Indias monthly export of spices data (in Rs. Crores) during the period April
to December 2006 is as follows:
Year

Apr

May

Jun

Jul

2000-01

137.95

129.95

123.51

123.28

2001-02

136.18

96.05

123.24

2002-03

148.44

177.88

2003-04

117.94

2004-05

Aug

Sep

Oct

123.97

124.68

119.42

109.23

129.99

118.62

110.75

106.18

152.74

156.13

2005-06

166.77

2006-07

187.32

2000

Nov

Dec

Jan

Feb

Mar

110.52

134.18

126.61

141.00

163.51

163.32

99.94

142.34

95.94

90.44

105.77

110.72

123.59

132.59

121.42

142.20

126.62

107.91

135.11

110.45

128.65

104.54

93.80

122.46

131.32

129.90

163.96

119.06

104.83

223.73

167.17

166.81

149.59

158.54

135.19

148.11

143.66

143.00

162.24

189.11

162.09

162.19

160.79

175.06

178.36

187.73

160.29

192.91

169.11

176.98

221.39

222.72

232.33

235.40

267.57

272.59

245.04

292.45

256.74

Fit GARCH nonlinear time-series model to above data using SAS Package.
Solution.
(i) Print Data
title 'Monthly Spices Data April 2000 to August 2006';
data spices;
input time y;
cards;
1 137.95
2 129.95
3 123.51
.
.
.
76 235.40
77 267.57
;
Proc print data=spices;
run;

127

11: Some Nonlinear Time-Series Models

(ii) Test for Heteroscedasticty

/* Test for Heteroscedasticty*/
ods html;
ods graphics on;
proc autoreg data=spices;
model y = / nlag=1 archtest dwprob method=ml;
output out=r r=yresid;
run; quit; ods graphics off;
ods html;
Table 1
Q and LM Tests for ARCH Disturbances
Order

Pr> Q

LM

19.0861

<.0001

35.1593

31.9060

<.0001

38.7479

Pr > LM

Order

Pr >Q

LM

<.0001

42.3082

<.0001

48.2483

<.0001

42.9105

<.0001

42.4390

<.0001

48.4019

<.0001

<.0001

46.2432

<.0001

42.4813

<.0001

48.7728

<.0001

39.6084

<.0001

46.4692

<.0001

10

42.4981

<.0001

48.9755

<.0001

42.1106

<.0001

47.5614

<.0001

11

42.7323

<.0001

48.9770

<.0001

42.1426

<.0001

47.6035

<.0001

12

42.7925

<.0001

49.7264

<.0001

(iii) Fitting of AR(1)-GARCH(1,1) model

Ods html;
Ods graphics on;
/*-- AR(1)-GARCH(1,1) model for the Y series --*/
proc autoreg data=spices ;
model y = / nlag=1 garch=(q=1,p=1) maxit=50 ;
output out=out_spices cev=vhat p=pred lcl=lcl ucl=ucl r=r_garch cpev=cpev;
run;
quit;
ods graphis off;
ods html;
Output of the AR(1)-GARCH(1,1)
AIC= 711.72, SBC=723.59

128

Pr > LM

11: Some Nonlinear Time-Series Models

Table 2
Variable

DF

Estimate

Intercept

149.4223

AR1

ARCH0

Standard
Error

t Value

Approx
Pr > |t|

11.2830

13.24

<.0001

-0.7888

0.0883

-8.93

<.0001

144.0878

96.9414

1.49

0.1372

ARCH1

0.8183

0.4530

1.81

0.0708

GARCH1

0.2112

0.2251

0.94

0.3482

From Table 2 Mean and Variance equations are:

y t = 149.42 0.789 y t -1 + t , where t = ht1 / 2t
(11.283) (0.088)
and
ht = 144.088 + 0.818 t2-1 + 0.211 h t -1
(96.941) (0.453) (0.225)
(iv) Graph
/*GRAPH of Fitted series along with data*/
proc sgplot data=out_spices;
scatter x=time y=y; series x=time y=pred;
xaxis label= 'Month (April,2000 To August, 2006)';
yaxis label=Indias Monthly export of spices (in Rs.Crore)';
run;quit;

Fig. Graph of fitted values along with data points

129

11: Some Nonlinear Time-Series Models

\
(v) Forecasts
/*Forecast data for four months*/
data b;
y = .;
do time = 78 to 81; output; end;
run;
data b;
merge spices b;
by time;
run;
proc print data=b;
run;
proc autoreg data=b ;
model y = / nlag=1 garch=(q=1,p=1) maxit=50;
output out=out_spices1 cev=vhat p=pred lcl=lcl ucl=ucl r=r_garch cpev=cpev;
run;
proc print data=out_spices1;
run;
Table 3
Forecasts of export of spices (in Rs. Crore) for fitted AR(1)-GARCH(1,1) model
Months

Actual
Value

One-step ahead forecast

(Static)

Multi-step ahead forecast

(Dynamic)

September 2006

272.59

242.62(48.66)

242.62(48.66)

October 2006

245.04

224.57(25.44)

222.94(50.81)

November 2006

292.45

210.16(16.69)

207.41(52.93)

December 2006

256.74

198.17(13.85)

195.17(55.51)

Figures in ( ) are corresponding standard errors.

REFERENCES
Abramovich, F., Bailey, T. C. and Sapatinas, T. (2000). Wavelet analysis and its statistical
applications. Statistician, 49, 1-29
Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods,
Springer, U.S.A.

130

11: Some Nonlinear Time-Series Models

Ghosh, H., Iquebal, M. A. and Prajneshu (2006a). On mixture nonlinear time-series modelling
and forecasting for ARCH effects. Sankhya, 68, 111-29
Ghosh, H., Paul, R. K. and Prajneshu (2010). Functional coefficient autoregressive
nonlinear time-series model for forecasting Indian lac export data. Model Assist. Statist.
Applns., 5, 101-08.
Ghosh, H. and Prajneshu. (2003). Nonlinear time-series modelling of volatile onion price data
using AR(p)-ARCH(q)-in-mean. Cal. Stat. Assn. Bull., 54, 231 47
Ghosh, H., Prajneshu and Paul, A. K. (2005). Study of nonlinear time series modelling in
agriculture. I.A.S.R.I. Project Report
Ghosh, H., Prajneshu and Sunilkumar, G. (2006b). Modelling and forecasting of bilinear timeseries using frequency domain approach. J. Combi. Info. Sys. Sci., 31, 217-33
Ghosh, H., Sunilkumar, G. and Prajneshu (2006c). Self exciting threshold autoregressive
models for describing cyclical data. Cal. Stat. Assn. Bull., 58, 115-32
Jansen, M. (2001). Noise Reduction by Wavelet Thresholding. Lecture Notes in
Statistics, Vol. 161, Springer-Verlag, U.S.A.
Percivel, D. B. and Walden, A. T. (2000). Wavelet methods for time series analysis. Cambridge
Univ. Press, U. K.
Tong, H. (1995). Nonlinear Time Series: A Dynamical System Approach. Oxford Univ. Press,
U. K.

131