International Journal of Forecasting 17 (2001) 403417 www.elsevier.

com / locate / ijforecast

Comparison of regime switching, probit and logit models in dating and forecasting US business cycles
a, b Allan P. Layton *, Masaki Katsuura
a

School of Economics and Finance, Queensland University of Technology, GPO Box 2434, Brisbane, Queensland 4001, Australia b Department of Economics, Meijo University, 1 -501 Shiogamaguchi, Tenpaku, Nagoya, Aichi 468 -8502, Japan

Abstract Three non-linear model specications are tested for their efcacy in dating and forecasting US business cycles, viz. a probit specication, a logit specication both binomial and multinomial alternatives and a markov, regime-switching specication. The models employ leading indicators compiled by the Economic Cycle Research Institute as putative explanators. They are tested within sample to determine their relative abilities to produce a business cycle chronology similar to the ofcial NBER chronology. They are also tested in a post-sample context to test their relative abilities in anticipating future turning points with the result that the regime-switching model with time-varying transition probabilities performs the best. 2001 International Institute of Forecasters. Published by Elsevier Science B.V.
Keywords: Business cycles; Regime switching model: logit and probit model; Quadratic probability score

1. Introduction Due to the longevity of the current US expansion some analysts and commentators have again questioned as many did in the late 1960s whether old conceptions of the business cycle are still relevant. It is certainly true that the current expansion has its very unique idiosyncratic features. Of course, this has been the case with all previous expansions also.
*Corresponding author. Tel.: 1 61-738-642-947; fax: 1 61-738-644-150. E-mail addresses: a.layton@qut.edu.au (A.P. Layton), katsuura@meijo-u.ac.jp (M. Katsuura).

Clearly, in the case of the current expansion, the impact which information and computer technology have had on unit production and distribution costs and labour productivity has been very important. Also important has been the role played by the FED in the conduct of monetary policy generally and, most particularly, during and after the nancial crisis of 1997 / 98. These factors, along with the historical changes to scal policy in the early to mid-1990s, and the subsequent budget surpluses, have all quite clearly helped create an environment of low ination and low interest rates over an unusually sustained period. This low ination period has also certainly been helped by the low commodity prices in 1998 / 99

0169-2070 / 01 / $ see front matter 2001 International Institute of Forecasters. Published by Elsevier Science B.V. PII: S0169-2070( 01 )00096-6

404

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

resulting from the Asian economic crisis and the very strong US dollar during the last few years. However, as noted at the outset, all other expansions have similarly had their unique features and, without exception, all have eventually come to an end. This will be the case with the current expansion also and, as has always been the case, it will be important to policymakers, and those on both Wall Street and Main Street to have some advance warning of when the inevitable contraction is likely to occur. More specically, despite the abovedescribed unique features of this particular expansion it is nonetheless likely to be the case that important information about the likelihood and timing of the next contraction will be found in movements in leading indicators which have been found over many years and in many countries to be so reliable in anticipating past business cycle turns. In meaningfully distilling this information it has been, and will remain, important to use the most appropriate modelling approaches. Against a background of the above considerations the objective of this paper is to provide some insight into the potential relative usefulness of two different classes of leading indicator-based models which have recently been proposed for predicting business cycle turning points; viz. the regime switching modelling specication and the logit / probit type formulation. Since Hamilton (1989) rst introduced his markov, regime-switching model formulation to macroeconomics the model, in one form or another, has been increasingly used by business cycle researchers to assist in the dating and forecasting of turning points in the business cycle. The model is conceptually appealing in that over time the variable of interest say some appropriate measure of the business cycle is regarded as having a certain probability of switching abruptly among a number of regimes. In the case of the business cycle we might envisage say, two regimes, one corresponding to expansions and the other to contractions. The

mechanism thought to be generating the observed data on the variable of interest is conceived as being regime-specic. This feature, and the fact that the regime can shift abruptly, means the model is intrinsically non-linear in nature. Lahiri and Wang (1994) provided the rst illustration of how such a model, as just described, could be used to predict business cycle turning points using the U.S. Commerce Departments composite leading index (CLI). In addition, one of the authors, Layton (1996), has found this basic model to be quite useful in dating the US business cycle using the coincident index as compiled by the Economic Cycle Research Institute (ECRI). In the basic model the transition probabilities which represent the likelihood of switching out of one regime into another are regarded as being xed through time. A useful extension of the basic model is potentially of more interest to proponents of the leading indicator approach to forecasting business cycle turning points. In this version of the model (see, for example, Filardo, 1994), the transition probabilities are regarded as variable over time and are specied as appropriate functions of the information contained in relevant leading indicators. Again, one of the authors, Layton (1998), following the suggestion of Diebold, Lee, and Weinbach (1994), found the variable transition probability version in which the transition probabilities were modelled as functions of ECRIs leading indexes was a potentially useful approach to characterising the US business cycle chronology. However, one limitation of this earlier paper was the lack of any post-sample testing of the model. A somewhat different modelling approach to forecasting turning points was suggested very recently by Estrella and Mishkin (1998). They modelled the probability of recession using a non-linear probit specication incorporating a number of alternative putative explanators. This is a fundamentally different approach to that

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

405

described above in that the model specication whilst non-linear in the sense that the model specication chosen is of probit form nonetheless conceives of only one time-invariant mechanism generating the probability of recession. Of course, another very common alternative to the probit specication used in applied forecasting is the logit specication. In the current paper, therefore, we thought it would be interesting to further investigate the out-of-sample performance of the variable transition probability markov model used earlier by Layton (1998) and to use, as yardsticks for comparison, model formulations which incorporate the logit and probit specications. As will be seen, not only does the markov formulation outperform the alternatives, but we will also argue that there are very important practical considerations which militate against the use of the alternative formulations in real-time business cycle turning point forecasting. It should be noted that a referee pointed out that Birchenhall, Jessen, Osborn, and Simpson (1994) have recently written a paper with similar objectives and found that, in contrast to our ndings, the logit specication was superior both within-sample and out-of-sample. However, there are many differences in the current analysis viz: the use of a different dependent variable in the regime switching model, a different set of explanatory variables in both types of models, lag structure, sample periods, the rule for judging business cycle phases, and the metrics used for model evaluation. In particular, in contrast to Birchenhall et al., in determining phase changes we use information from a number of successive months to determine turning points and also make use of traditional lead / lag analysis to compare models. Furthermore, due to the differing nature of the dependent variable in the two different types of models we use quadratic probability score (dened later) as a core metric of in-sample and out-of-sample model evaluation. Birchenhall et

al. (1994) resolved this problem in quite a different way by using an adjusted log likelihood for the regime-switching models in order to try to construct an appropriate benchmark against which the performance of the logit models could be judged. The rest of the paper is organised as follows. In the next section we briey describe the model specications. As the logit and probit formulations are quite well-known, the greater emphasis will be on the markov formulation. The third section contains the empirics and the fourth section contains the conclusions. 2. Model specications

2.1. The regime switching model

Hamilton (1989, 1990) proposed a markovian, regime-switching, modelling approach as a useful characterisation for certain economic and nancial time series. In simple terms a time series is conceived of being in one of a number of different states (or regimes). The likelihood of observing various values of the series then depends upon the particular regime in which the series is, along with a specied probability rule associated with that regime. In the context of the business cycle it is natural to dichotomise between two regimes, viz. contractions and expansions in economic activity. For each regime the simplest probability rule to govern the likelihood of various observations is the normal density function with different means and variances for contractions and expansions. Hamilton also suggested the use of a markov-type probability rule to govern the likelihood of switching from one regime into the other. The simplest such rule is that the switching probability depends only on the current period regime (rather than several earlier periods), and remains constant regardless of the length of time the series has been in a particular regime.

406

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

More recent work relaxes this invariance restriction. For example, Diebold et al. (1994) suggest allowing the transition probabilities to be modelled as logistic functions of whatever determinants one thinks may potentially be driving them. This is the specic regime-switching model employed throughout this paper and can be represented as follows. Let Yt and St denote the growth rate and regime state respectively in period t. Then: (Yt 2 mi ) 1 f (Yi uSt 5 i ) 5 ]]] ]] exp 2 ]]] 2s 2 2psi i

that Yt will be in contraction, for each time period. If P(St 5 1) . 0.5, the economy may be judged prima facie to be in contraction, otherwise in expansion. To obtain estimates of the parameter vector ( m1 , m2 , s1 , s2 , b1 , b2 ), maximum likelihood estimation (MLE) is used.

2.2. The logit model

Here, it is assumed that we know, with absolute certainty, the state for each sample observation in the time series. For the current analysis, the ofcial US business cycle chronology as determined by the National Bureau of Economic research (NBER) can be used to assign the relevant state for each observation. Thus, here we assign St 5 0 (for periods of contractions) or 1 (for periods of expansions),1 and the variable S is modelled directly as the dependent variable in a model incorporating the logistic function specication and of the form 1 P(St 5 0) 5 ]]]]], (1 1 exp(2Xt a )) where X contains the explanatory variables and a is a set of parameters to be estimated. This model is actually referred to as the binomial logit model and again the parameters of the model are estimated by MLE. Forecasts of S from this model have the interpretation of probability forecasts of S being either 0 or 1, conditional on the values of the explanatory variables in the model. An important extension of this basic binomial logit model is the so-called multinomial logit model. In this extension the dependent variable, S, is allowed to have more than two values. For example, in the present analysis, in order to facilitate comparison with the markov model, it
1

where i 5 1, 2 and 1 denotes contraction and 2 denotes expansion; and Pt (St 5 1uSt 2 1 5 1) 5 p11 t , Pt (St 5 1uSt 2 1 5 2) 5 1 2 p22 t Pt (St 5 2uSt 2 1 5 1) 5 1 2 p11 t , Pt (St 5 2uSt 2 1 5 2) 5 p22 t where 1 piit 5 ]]]]]]]]] (1 1 exp(2Xt 2 1 bi ));i 5 1,2 and Xt 2 1 5 (1, x 1, t 2 1 , x 2, t 2 1 , . . . , x k 2 1, t 2 1 );

bi 5 ( bi 0 , bi 1 , . . . , bi , k 2 1 )9
and where k 2 1 is the number of putative determinants of the transition probabilities. It is worth pointing out that in this model, whereas we observe Yt , we do not observe St directly. In fact, an important advantage of this model over those below is that we do not need to observe St directly in order to estimate the model. However, an important output of the model is the estimate of the likelihood of St taking the value 1 or 2 in each period. Other very useful outputs of this model are the estimates of the transition probabilities p11 t and p22 t , and P(St 5 1), the unconditional probability

Note that in regime switching model, 1 denotes contraction and 2 denotes expansion.

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

407

is convenient to dene two additional states for S, viz. St 5 0 when the observation corresponds to a contraction; St 5 1 when the observation corresponds to a contraction and is, in addition, an actual trough date; St 5 2 when the observation corresponds to an expansion; St 5 3 when the observation corresponds to an expansion and is, in addition, an actual peak date, all dates again as determined by the NBER. The multinomial logit model is specied by exp(Xit ai ) P(St 5 i ) 5 ]]]] 5 pit , i 5 0, 1, 2, 3 3 exp(Xit ai )
i 50

and the probability that the economy will be in contraction is dened as p0 t 1 p1 t , and p2 t 1 p3 t , the probability that the economy will be in expansion.

2.3. The probit model

As with the binomial logit model it is assumed that we know, with absolute certainty, the state for each observation in the time series. Again, the ofcial US business cycle chronology as determined by the NBER can be used to assign the relevant state for each observation. The difference between the probit model and the logit model described above is that, instead of the logistic function, a cumulative standard normal distribution functional form is used in the model. Thus, here again we assign St 5 0 (for periods of contractions) or 1 (for periods of expansions), and the variable S is modelled directly as the dependent variable in a model of the form P(St 5 0) 5 F (Xt a )

where F ( ? ) denotes the value of the cumulative standard normal distribution. This model is actually referred to as the binomial probit model and, again, the parameters of the model are estimated by MLE. The functional forms of both the logit and probit models guarantee the estimated probabilities which result from the model are between 0 and 1 and that they sum to one as required. As to which functional form is to be preferred there is no compelling theoretical reason why either should be regarded as superior. The differences between them occur in their behaviour in the extremities of the 01 probability range (the logistic has fatter tails) but this does not imply one should be preferred over the other. For further details the interested reader is referred to, for example, Greene (1997). In theory, the binomial probit model can be extended to a multinomial version but, due to estimation complexity, in practice, if it is desired to allow for more than two states, the multinomial logit model is almost always used. This is the case here also so that we do not attempt to estimate a multinomial probit.

2.4. Model evaluation metrics

Apart from more usual measures of evaluating alternative models such as R2 , log likelihood, or information criteria, we felt that other metrics could be usefully employed for comparing models due to our focus on dating and forecasting business cycle phases. In this regard, we can use the ofcial dates of the US business cycle chronology, as determined by the NBER, as a benchmark for comparison. It is worth mentioning that an alternative would be to use the turning points as determined by, say, the application of the Bry and Boschan (1971) algorithm to the ECRI coincident index which is used in the empirics below and which is constructed for the purpose

408

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

of representing the cyclical movements in the economy. Its turning points are reasonably similar to the ofcial NBER chronology but certainly do not represent a perfect match. Though representing a reasonable proxy for the business cycle and certainly better than other commonly-used single series proxies the business cycle is the widely-accepted ofcial NBER turning point chronology and not the turning points in any single macroeconomic time series. It should be emphasised then that, as far as our purposes are concerned, all models are being investigated for their ability to predict the ofcial chronology of the US business cycle as represented by the NBER dates. As such, it is therefore more appropriate that it be this set of dates which is used in the evaluation which follows. For the same reasons as just explained, it is also this set of dates which form the basis of the dependent variable in the logit and probit models described in the previous section. As described above, all of the models estimate the probabilities that the economy is in contraction or expansion P(St 5 i ). When these probabilities are more than 0.5, the economy could therefore be judged to be in contraction or expansion and, in this way, we can date the turning points as derived from the models. One simple method to evaluate the models is thus to compare the dates derived from the various models with the ofcial NBER dates, and to check for the number of months difference as well as any false signals in evidence. A method of quantifying the degree of this correspondence was introduced into business cycle analysis by Diebold and Rudebusch (1989) and is known as Quadratic Probability Score (QPS). QPS is dened as 1 QPS 5 ] (P 2 Dt )2 T t 51 t where Dt takes the value 1 during expansion

and 0 during contraction as identied by the NBER chronology and Pt is the model-derived probability for the corresponding observation. The closer this measure is to zero, the better is the t to the ofcial chronology. Whilst QPS is useful for evaluating performance in dating, we need another metric for evaluating forecasts of phase switching probabilities. We dene two measures of probability of a turning point occurring in the following 6 months as follows:
A) 6 ( A) 6 p( 12 T 5 1 2 p 11 T , p 21 T 5 1 2 p 22 T

and p 12 T 5 1 2 p11, T 1 1 3 p11, T 1 2 3 ? ? ?

B) 3 p11, T 1 6 , p ( 21 T 5 1 2 p 22, T 1 1 (B )

3 p22, T 1 2 3 ? ? ? 3 p22, T 1 6
l) (l ) where p ( 12 T , ( p 21 T ), l 5 A, B, denote the probability that a trough (peak) will occur within the period T 1 1, T 1 2, . . . , T 1 6 at the time T. Measure A assumes that the switching probability will take the same value for the next 6 months, while measure B requires future probabilities to be estimated from the model. However, as we explain in the next section, since we will use lagged leading indicator as an explanatory variable in the time-varying transition probability model, the model can provide estimation of these probabilities. To evaluate the above probabilities, we dene another quadratic probability score, denoted by QPS6, as follows:

1 l) 2 QPS 6 5 ] (TPP 6 ( t 2 D 6 t ) ; l 5 A, B T t 51
l) (l ) (l ) where TPP 6 ( t 5 p 12 t 3 Pt 2 1 1 p 21 t 3 (1 2 Pt 2 1 ), D 6 i 5 1 if a turning point occurs in one of the periods t 1 1, t 1 2, . . . , t 1 6, otherwise 0, and Pt 2 1 5 1 during contraction and Pt 2 1 5 0 during expansion, dates as determined by the

O
T

O
T

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

409

Table 1 Index components a Current gauge Personal income (real) GDP (real) Industrial production Manufacturing and trade sales Employees in nonfarm sector Unemployment rate (inverted) Short run gauge New building permits Ratio, price to unit labour cost Average workweek, manufacturing Average weekly initial claims (inverted) New orders, consumer goods and materials Contracts and orders, plan and equipment Stock price index, S and P 500 Change in industrial material prices Change in business inventories Corporate prots after taxes Net business formation Change in consumer installment credit Long run gauge New building permits Ratio, price to unit labour cost Output / manhour, manufacturing, growth rate Deated money supply M 2 Dow Jones 20 bond average Services CPI index, growth rate, (inverted)

Note: Quarterly series like GDP are interpolated by ECRI to produce monthly data for incorporate in the indexes.

ofcial NBER chronology.2 TPP6 was designed to represent the model-derived probability of a turning point occurring sometime in the next 6 months. QPS6 should therefore be a measure of the closeness of the model probabilities to the true turning point probabilities of one or zero.

3. The empirics

3.1. Data and preliminary analysis

Data for the analysis are monthly, span the period February 1949July 1999, and are supplied by ECRI. In particular, we use ECRIs US Current Gauge (CG a composite index of coincident business cycle indicators), Short Range Gauge (SRG a composite index of short leading business cycle indicators), and the Long Range Gauge (LRG a composite index of long leading business cycle indicators). In the various models estimated, the actual data used were month-to-month growth rates of the various composites.
2

The reason for using, rather than using the model derived probability, is discussed in Layton (1998).

Components of these indexes are provided in Table 1. Readers familiar with other common business cycle-type indexes such as those of the Conference Board will note some signicant differences. For instance, CG includes the additional coincident indicators of GDP and unemployment. On the other hand, of the ten components included in the Conference Boards leading index, ve are included in SRG, two are in LRG, two others have no close analogy in either SRG or LRG, while the Conference Boards measure of monetary conditions tightness (interest rate spread) has an alternative but analogous measure in LRG (viz bond prices). The other ten components in SRG and LRG do not have corresponding components in the Conference Boards leading index. Of course, the other important feature is the separation by ECRI of leading indicators into long leading and short leading. Initially, a lead / lag analysis of the composites was conducted in relation to the NBER chronology using the Bry and Boschan (1971) turning point algorithm to determine the dates of the turning points in the various composites. This analysis is provided in Table 2. As expected, but nonetheless reassuring, we observe

410

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

Table 2 Lead-lag table for ECRI coincident gauge (CG), short range gauge (SRG) and long range gauge (LRG)a NBER ofcial dates Peak Jul-53 Aug-57 Apr-60 Dec-69 Nov-73 Jan-80 Jul-81 Jul-90 false signal mean median Trough Oct-49 May-54 Apr-58 Feb-61 Nov-70 Mar-75 Jul-80 Nov-82 Mar-91 false signal mean median Total false signal mean median
a

CG 0 0 22 22 0 0 0 21 0 2 0.6 0

SRG 24 2 21 2 10 27 28 2 15 23 2 15 3 2 10.4 29

LRG 28 2 30 2 17 2 10 2 11 2 29 2 10 2 20 4 2 16.9 2 14

point, and they display a median overall lead of 4 (SRG) and 9.5 (LRG) months. This represents a very good lead / lag analysis and suggests the system is quite suitably reliable in the traditional leading indicator analysis sense. In the various models estimated in the paper, these identied leads play an important part in determining the form of the constructed explanators.

3.2. In-sample results

Two regime switching models were estimated with results provided in Table 2. One is a constant transition probability version while the other is a variable transition probability model in which the explanator is a function of LRG. The particular function chosen throughout this analysis for all models was that of a moving sum of movements in LRG over lags six to twelve; o 12 s 5 6 LRG t 2 s Such a specication whilst somewhat arbitrary economises on the use of parameters and reduces the potential problem of multicollinearity in the estimation. It nonetheless should capture the proposition that both the amplitude and duration of a phase shift in the leading index should be informative as far as the future course of the business cycle is concerned. Whilst selecting seven as the span of the moving average is also somewhat arbitrary the end points of the span were chosen so as to centre on the identied median lead of LRG noted in Table 2. Models were initially estimated incorporating the SRG on its own, the LRG on its own, and both composites combined. However, the model using the LRG on its own was found to be preferable. It produced a model with a similar likelihood but with all coefcients taking the expected signs whilst using SRG produced a model with one coefcient with an inappropriate sign. Finally, within the markov framework for identifying upcoming turning points (see notes to Table 6), on an

0 0 0 22 0 0 21 1 0 0 2 0.2 0

24 24 22 22 0 21 22 23 22 0 2 2.2 22

missing 25 22 28 29 22 23 2 12 22 0 2 5.4 24

0 2 0.4 0

3 2 6.1 24

4 2 11.1 2 9.5

Note: Sample; Feb.1949Jul.1999. Turning points for each gauge are determined by the Bry and Boschan (1971) Method. Figures are numbers of months.

that the mean lead of CG is less than a month, its median lead is zero months, and that it has no false turning points. The two leading indexes produce a small but acceptable number of false turning point signals (these are all at peaks). Only one turning point is missed. The indexes do not lag at any turning

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

411

in-sample basis SRG produced one more false signal than that using LRG.3 This is consistent with the earlier work mentioned in the introduction (Layton, 1998). Also, another reason for preferring LRG is its longer average lead. This is an advantage in practice in real time forecasting of turning points. Therefore, in what follows, LRG is the chosen leading index explanator in all models estimated. A couple of points about Table 3 are noteworthy. First, looking at the rst column of the table, all parameter estimates of the constant transition probability model are statistically signicant and of the expected sign. Secondly, the parameter estimates of the variable transition probability model are also of the expected sign but the coefcient of LRG is statistically insignicant for state 1. This may be interpreted as suggesting that LRG may be more informative with respect to anticipating future peaks in the business cycle during expansions rather than anticipating future troughs during contractions. Also provided in the table is a measure of how close the model-derived probabilities, P(St 5 1), of contraction are to the actual probabilities (i.e. either 0 or 1 as determined by the NBER dates) for each period over the sample. This measure is QPS, described earlier, and as noted there, the closer this measure is to zero the better. As can be seen from the table both models produce a very similar QPS. This is to be expected since LRG would not be expected to provide very much if any additional precision in terms of ex post dating of the business cycle over and above that provided by CG itself. The real value of the leading index could be expected to be in the real-time ex ante forecasting of future business cycle turning points (see later).
3

Table 3 Results of estimation for coincident gauge in regime switching models a Parameter p11 p22 m1 m2 s2 1 s2 2 b10 b11 b20 b21 Log likelihood QPS
a

Model RS1 0.8823 (0.0351) 0.9719 (0.0071) 2 0.4014 (0.0753) 0.4327 (0.0219) 0.3037 (0.1269) 0.1930 (0.0224)

Model RS2

2 446.56 0.034020

2 0.4233 (0.0780) 0.4286 (0.0227) 0.3021 (0.1247) 0.1938 (0.0223) 1.8232 (0.3683) 2 0.0029 (0.1170) 3.3507 (0.2944) 0.2036 (0.1023) 2 443.41 0.032339

Notes: Sample period is Feb. 1949Jul. 1999. The numbers in parentheses are asymptotic standard errors. Dependent variable is CG t . Model RS1: Constant transition probability model. Model RS2: Varying transition probability model using LRG. bij : parameter for i -th phase (i 5 1, 2) and j -th explanatory variable (i 5 1 for contraction and i 5 2 for expansion; j 5 0 for constant term and j 5 1 for LRG). LRG t is moving sum of previous 6 to 12 months month-to-month growth rates, viz. LRGt 5 LRGt 2 6 1 LRG t 2 7 1 ? ? ? 1 LRG t 2 12 . For QPS, see Diebold and Rudebusch (1989) and the text of the paper. In general, QPS 5 T 2 1 o(MGPt 2 Dt )2 where say MGPt 5 the model-generated probability of contraction and Dt 5 the actual probability of 0 or 1 as determined by the NBER chronology of contractions.

This is to be distinguished from the results of the traditional lead / lag turning point analysis using Bry and Boschan on CG, SRG and LRG as reported in Table 2.

Table 4 contains the estimation results for various binomial logit and probit models. For each functional form three models are estimated, viz. one in which CG is the explanatory variable, one in which LRG is the explanatory variable, and one in which both appear as explanatory variables. Broadly speaking the essence of the table is that any model which excludes CG is a relatively poor model in terms of both R 2 and QPS. It would seem that the logit specication marginally outperforms the probit specication although there is little dif-

412

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

Table 4 Results of estimation of binomial probit and logit model a

(a) Parameter estimates Parameter const. CG LRG Log likelihood R2 QPS 2 145.6290 0.4431 0.066586 Model BNP1 0.9245 (0.0820) 2.4516 (0.2104) 0.1568 (0.0197) 2 223.5120 0.1452 0.114666 Model BNP2 0.9388 (0.0668) Model BNP3 0.9139 (0.0884) 2.3816 (0.2199) 0.1415 (0.0249) 2 127.4780 0.5125 0.056674 2 141.4583 0.4590 0.064448 Model BNL1 1.6280 (0.1606) 4.9854 (0.4965) 0.2779 (0.0358) 2 224.1576 0.1427 0.115022 Model BNL2 1.6024 (0.1230) Model BNL3 1.6571 (0.1782) 4.9292 (0.5242) 0.2787 (0.0500) 2 122.3424 0.5321 0.054408

(b) dPi /dxj regime i50 i51 variable BNP1 CG LRG CG LRG
a

BNP2

BNP3 2 0.2866

BNL1 2 0.3421

BNL2

BNL3 2 0.2922

2 0.3328 2 0.0319 0.3328 0.0319

2 0.0170 0.2866 0.0170 0.3421

2 0.0314 0.3421

2 0.0165 0.2922 0.0165

Note: Sample period is Feb.1949Jul.1999. Dependent variable is equal to 0 for contractions and 1 for expansions; the dates correspond to the ofcial NBER business cycle chronology. The numbers in parentheses are asymptotic standard errors. CG is current value of Coincident Gauge. LRGt is moving sum of previous 6 to 12 months month-to-month growth rates, viz. LRG t 5 LRG t 2 6 1 LRG t 2 7 1 . . . 1 LRG t 2 12 . BNP13 denote Binomial Probit and BNL13 denote Binomial Logit. Model BNP1 and BNL1 uses CG as explanatory variable. Model BNP2 and BNL2 uses LRG as explanatory variable. Model BNP3 and BNL3 uses CG and LRG as explanatory variables.

ference between the two specications. Finally, in terms of QPS none of the models comes close to either of the two regime switching models. Table 5 contains the estimation results for the multinomial logit specication. Again three alternative models were estimated. It would appear from the table that there is no signicant improvement in using the multinomial specication as compared with the binomial specication in terms of QPS.4
4

A nested logit model specication is also potentially applicable as one of the multinomial logit models. This would result from assuming a tree structure in which the rst branch is whether the economy is in contraction or expansion, and then the second sub-branch represents whether the period corresponds to a turning point or not. However, such a model was not found to be statistically superior to the non-nested multinomial logit model reported here.

As a measure of closeness to implied NBER business cycle phases, QPS is quite a good metric and one which incorporates every observation. Also important, however, is the performance of each of the various models in terms of the degree of date correspondence between implied turning points from the model and the ofcial NBER turning points. This is presented in Table 6. As is clearly evident from the table, the regime switching models are quite superior on this basis. For example, model RS2, which incorporates LRG in the regime-switching context, results in no turning points being missed, only one false signal being given, and with identied turning point dates which are very close to those of the NBER. Model RS1 performs similarly. However, the logit and probit models using the same identication criteria perform quite poorly with a large number of missed turning points and quite a few identied

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

413

Table 5 Results of estimation for multinomial logic model a (a) Parameter estimates Regime Parameter i51 const. CG LRG const. CG LRG const. CG LRG Model MNL1 2 2.5739 2 0.6036 1.6790 4.9686 2 2.1110 3.8591 2 211.00 0.064445 (0.4897) (0.5425) (0.1645) (0.4992) (0.4401) (1.0301) Model MNL2 2 2.1696 0.0672 1.6910 0.2886 2 2.2689 0.1224 2 293.39 0.115008 (0.3595) (0.0975) (0.1295) (0.0375) (0.3729) (0.1032) Model MNL3 2 2.4692 2 0.5451 0.0640 1.7131 4.9133 0.2887 2 1.9656 3.9620 0.1379 2 190.58 0.054415 (0.5111) (0.5505) (0.1114) (0.1837) (0.5276) (0.0511) (0.4483) (1.0504) (0.1082)

i52

i53

Log likelihood QPS (b) dpi / dx j Regime i50 i51 i52 i53
a

Variable CG LRG CG LRG CG LRG CG LRG

MNL1 2 0.3083

MNL2 2 0.0292

MLN3 2 0.2614 2 0.0160 2 0.0308 2 0.0005 0.2945 0.0179 2 0.0024 2 0.0014

2 0.0337 2 0.0023 0.3477 0.0328 2 0.0056 2 0.0014

Note: Sample period is Feb. 1949July 1999. Dependent variable is equal to 0 when the observation corresponds to a contraction, (1) when the observation corresponds to a contraction and is, in addition, an actual trough date, (2) when the observation corresponds to an expansion, and (3) when the observation corresponds to an expansion and is, in addition, an actual peak date; dates again as per NBER chronology. Model MNL1 uses CG as explanatory variable. Model MNL2 uses LRG as explanatory variable. Model NML3 uses CG and LRG as explanatory variables. In calculating QPS, the probabilities of contraction and expansion are dened as P0 1 P1 and P2 P3 , respectively, where the P1 are the model-generated probabilities of the four different states dened above.

turning points which diverge by too large a number of months from the NBER dates.

3.3. Out-of-sample results

Now, to get an idea of the usefulness of the models in an out-of-sample context, data up to January 1979 5 were used to estimate the models and a one-step ahead forecast was calculated.
5

Jan. 1979 was selected because it was twelve months before the third most recent business cycle peak.

One observation was then added and the process repeated one observation at a time through until the end of the whole sample. As a summary measure of closeness to the NBER turning point dates QPS was again calculated from February 1979 to July 1999 for each of the 11 models. The results are provided in Table 7. As is clearly evident from the table the two regime switching models again perform relatively better than the logit and probit specications, and of these, the regime switching model incorporating LRG is slightly superior. On this basis we

414

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

Table 6 Lead-lag months to NBER dates by RS, BNL and MNL models a
Model ofcial dates RS1 Peak Nov-48 Jul-53 Aug-57 Apr-60 Dec-69 Nov-73 Jan-80 Jul-81 Jul-90 False signal Trough Oct-49 May-54 Apr-58 Feb-61 Nov-70 Mar-75 Jul-80 Nov-82 Mar-91 False signal Total false signal 21 21 0 21 21 11 21 0 21 0 2 21 21 0 21 21 11 21 0 21 0 1 21 23 21 23 21 21 23 22 23 3 3 Missing Missing Missing 2 10 23 28 22 Missing Missing 0 1 21 22 21 23 21 21 22 24 21 3 3 21 22 21 23 21 21 23 22 23 3 3 Missing Missing Missing 2 10 23 28 22 Missing Missing 1 2 21 22 21 23 21 21 22 24 21 3 3 21 22 21 23 21 21 23 22 23 2 2 Missing Missing Missing 2 10 23 28 22 Missing Missing 0 1 21 22 21 23 21 21 22 24 21 3 3 na 0 21 0 21 0 0 11 0 2 na 0 0 21 21 0 0 11 0 1 na 21 21 14 Missing Missing 0 0 0 0 na Missing Missing Missing 15 14 Missing Missing Missing 1 na 0 21 Missing 16 21 0 11 14 0 na 21 21 14 Missing Missing 0 0 0 0 na Missing Missing Missing 15 14 Missing Missing Missing 1 na 0 21 Missing 16 21 21 11 14 0 na 21 21 14 Missing 7 0 0 0 0 na Missing Missing Missing 5 4 Missing Missing Missing 1 na 0 21 Missing 16 21 21 11 14 0 RS2 BNP1 BNP2 BNP3 BNL1 BNL2 BNL3 MNL1 MNL2 MNL3 Regime switching model Binomial probit model Binomial logit model Multinomial logit model

a Note: Turning points are judged to occur at time t when the data point probabilities satisfy the following conditions for four successive months. Conditions are Pt 1 1 , Pt 1 2 , Pt 1 3 . 0.5 and Pt , 0.5 for trough, and Pt 1 1 , Pt 1 2 , Pt 1 3 , 0.5 and Pt . 0.5 for peak. P is the probability that the economy is in an expansion, viz. P 5 P1 for binomial logit and probit models, P 5 P2 1 P3 for multinomial logit model, and P 5 P(s t 5 2uFt ) in regime switching model.

concluded that the markov-type model was superior to the other specications in this particular investigation. Given this, we were interested to see how the two RS models would perform in anticipating future turning points in an out-of-sample context when the horizon was extended to say 6 months a common horizon for business cycle leading indicator analysis. The results are provided in Table 8 where again the out-of-sample period analysis commenced in February 1979 and proceeded in the same fashion as described above. To do this analysis the transition probability estimates were used to construct the

metric, denoted QPS6, as described in the previous section. As is evident from the table the model incorporating LRG improves the degree of correspondence. Whilst seemingly marginal, the improvement is statistically signicant using the Wilcoxon signed rank test.6 Looking at the
6

The authors are grateful to an anonymous referee for the suggestion of this test. The reader is referred to Diebold and Mariano (1995) for further details. The test statistic was found to be 2.537 and, under the null of no improvement, this test statistic is distributed as N (0,1). The null is therefore rejected at the 5% signicance level.

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

415

Table 7 QPS calculations for out-of-sample, one-step-ahead model forecasts a QPS RS1 RS2 BNP1 BNP2 BNP3 BNL1 BNL2 BNL3 MNL1 MNL2 MNL3
a

0.032033 0.030255 0.057388 0.092189 0.048535 0.054759 0.092070 0.045670 0.054759 0.091990 0.045688

Note: First a one-step-ahead forecast based on the sample Feb. 1949Jan. 1979 inclusive is calculated. Models were then repeatedly re-estimated in a step-wise fashion using one additional observation and one-stepahead forecasts calculated each time through until Jul. 1999. QPS is calculated for the period; Feb.1979Jul. 1999. Table 8 QPS6 comparison of forecasted probabilities for the occurrence of turning points in the next 6 months a RS1 P PB
A

Finally, model RS2, incorporating LRG, was then used to project forward the implied modelderived probability of a turning point occurring in the US in other words a switch from the current expansion into a contraction sometime during the 6 months beyond the end of the sample period available for the analysis; i.e. during the period August 1999January 2000. This is provided in Table 9. For comparison, corresponding probabilities are provided for the last 6 months of the data period as well. Looking at the probability of remaining in the expansion (in two successive months) it is evident that, in the early part of 1999, this probability was around the long run average for the entire sample of 0.967 which in turn is similar to the constant transition probability model parameter estimate of 0.972. However, movements in LRG in the rst half of the year were such as to make it relatively more likely that the economy will remain in an expansion in
Table 9 Recent and forecast transition probabilities by regime switching model a p22 t Jan-99 Feb-99 Mar-99 Apr-99 May-99 Jun-99 Jul-99 Aug-99 Sep-99 Oct-99 Nov-99 Dec-99 Jan-00 overall average
a

RS2 0.106352 0.103654

0.109884

p21 t 0.0330 0.0275 0.0238 0.0395 0.0370 0.0342 0.0261 0.0173 0.0187 0.0182 0.0149 0.0175 0.0212 0.0326

pA 21 T 0.1822 0.1538 0.1348 0.2150 0.2025 0.1885 0.1469 0.0993 0.1069 0.1044 0.0860 0.1004 0.1207 0.1713

pB 21 T 0.1799 0.1742 0.1655 0.1611 0.1425 0.1227 0.1075 0.1030 0.1736

Notes: Sample periods are the same as in Table 7, and QPS6 is calculated for the period; Feb.1979Jul. 1999. QPS6 is calculated by QPS6 5 1 / T *o(TPP 6 j 2 D 6)2 ; j 5 6 A 6 B A,B, where p A 12 T 5 1 2 p 11 T , p 21 T 5 1 2 p 22 T , p 12 T 5 1 2 B * T 1 1 p 11, * T 1 2 . . . *p11, T 1 6 ), ( p 11, p 21 T 5 1 2 j j * * * ( p 22, T 1 1 p 22, T 1 2 . . . *p22, T 1 6 ), TPP 6 5 p 12 T PT 2 1 1 j *T (1 2 PT 2 1 ), D 6T 5 1 if turning point occurs in one of p 21 periods T 1 1, T 1 2, . . . , T 1 6, PT 2 1 5 1 during contraction and PT 2 1 5 0 during expansion, dates as determined by the ofcial NBER chronology.

difference between p ( A) and p (B ) , QPS6 for p (B ) is smaller. This is not too surprising in that p ( A) may be viewed as an approximation of the probability p (B ) . However, the improvement is again marginal, and either p ( A) or p (B ) could be used as the forecast probability of a turning point occurring in the next 6 months.

0.9670 0.9725 0.9762 0.9605 0.9630 0.9658 0.9739 0.9827 0.9813 0.9818 0.9851 0.9825 0.9788 0.9674

Note: Sample period is Feb. 1949Jul.1999. RS2 model (using LRG) is employed. Overall average is that calculated from the entire sample period. See notes in B B Table 8 relating to p A 21 T and p 21 T . p 21 T represents the probability of occurrence of a peak sometime during the period T 5 1, T 1 2, . . . , T 1 6.

416

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

the near future. Specically the model is estimating a corresponding probability through until Jan 2000 of around 0.98. While this does not appear to be much of an increase it reduces the probability of a phase shift occurring some time in the following six month period from around 18% to around just 10% (see P A and P B columns of the table). The model thus suggests that, on the basis of recent movements in LRG, the likelihood of a phase shift out of expansion occurring over the six month forecast horizon through until Jan 2000 to be historically quite low.7

4. Conclusions As noted in the introduction, whilst the current US expansion certainly has its unique features, this has also been the case for all other eight expansions in the post-World War II period and all of those have eventually come to an end. This will certainly be the case with the current expansion also and, as always been the case, it will be important for all the various stakeholders to have some advance warning of when the next inevitable contraction is likely to occur. In the paper we have therefore tested, in the context of the traditional leading indicator approach to business cycle analysis, the relative performance of a number of alternative nonlinear model specications for forecasting the probability of phase shifts. Specications investigated were the logit, probit and markov regime switching formulations. On the basis of within-sample goodness of t measures and
7

A referee suggested it would be interesting to see how the estimated probit and logit models would have fared in this forecasting period. Using estimated model BNL2 the predicted probability of expansion exceeds 0.9 for the months August to December 1999, and is 0.86 in January 2000.

turning point dating, as well as out-of-sample turning point forecasting, our analysis suggests the markov regime switching specication performs relatively better than the other models. Within the regime switching framework, the variable transition probability model including LRG as putative explanator, is slightly better than the xed transition probability model in terms of both in- and out-of-sample results. Using the model incorporating LRG we forecast the probability of switching into contraction sometime during the six month period from August 1999 to January 2000 to be very low. Apart from the empirically superior performance of the markov specication found in this particular instance we feel there is also a very real practical limitation of the logit and probit specications as far as their use in real time business cycle phase shift forecasting is concerned. Their estimation requires exact knowledge of the regime state of the economy for every observation in the estimation period so as to assign values to the dependent variable in the model. In real time estimation, where the analyst would desirably want to update the estimation of the model every month as new information from leading and coincident indicators became available, this would be quite problematical. In the vicinity of a turning point, if re-estimation of the parameters were desirable, a view would have to be taken as to the state of the economy in recent months, possibly right up to the current period. Of course, this is precisely what would not be known in real time and indeed such an assignment would be the objective of the whole modelling exercise. We feel this issue represents a fundamental problem with the use of these types of models to forecast business cycle turning points. The markov-type model, on the other hand, does not suffer from this drawback. In effect the dependent variable in the markov model is the coincident index or whatever measure is being used to represent the

A.P. Layton, M. Katsuura / International Journal of Forecasting 17 (2001) 403 417

417

current level of economic activity and estimates of phase probabilities are produced as an output of the model. We believe this represents a signicant advantage of the Markov family of models as far as business cycle phase shift analysis is concerned. As a nal point though, it is worth re-iterating that the motivation for the paper was to further investigate the out-of-sample forecasting performance of the markov-type model used earlier in Layton (1998), and to use as benchmarks for comparison, alternative model formulations which incorporate logit and probit specications, such formulations being recently proposed by Estrella and Mishkin (1998). It is, of course, the case that the conclusion of the paper about the empirical superiority of the markov-switching type of model over the alternatives studied is obviously at least partially dependent upon the choice of indicators used, the sample period under study, the metrics used, and so on. References
Birchenhall, C. R., Jessen, H., Osborn, D. R., & Simpson, P. (1994). Predicting U.S. business cycle regimes. Journal of Business and Economic Statistics 17, 313 323. Bry, G., & Boschan, C. (1971). Cyclical analysis of time series selected procedures and computer programs. In: Technical paper, Vol. 20, NBER Columbia University Press, New York.

Diebold, F. X., & Rudebusch, R. (1989). Scoring the leading indicators. Journal of Business 62, 369402. Diebold, F. X., Lee, J. H., & Weinbach, G. C. (1994). Regime switching with time-varying transition probabilities. In: Hargreaves, C. (Ed.), Nonstationary timeseries analysis and cointegration, Oxford University Press. Diebold, F., & Mariano, R. (1995). Comparing predictive accuracy. Journal of Business and Economic Statistics 13, 253263. Estrella, A., & Mishkin, F. S. (1998). Predicting U.S. recessions: nancial variables as leading indicators. The Review of Economics and Statistics 80, 4560. Filardo, A. J. (1994). Business-cycle phases and their transitional dynamics. Journal of Business and Economic Statistics 12, 299308. Greene, W. H. (1997). Econometric analysis, 3rd ed, Prentice Hall, New Jersey. Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57, 357384. Hamilton, J. D. (1990). Analysis of time series subject to changes in regime. Journal of Econometrics 45, 3970. Lahiri, K., & Wang, J. G. (1994). Predicting cyclical turning points with leading index in a Markov switching model. Journal of Forecasting 13, 245264. Layton, A. P. (1996). Dating and predicting phase changes in the U.S. business cycle. International Journal of Forecasting 12, 417428. Layton, A. P. (1998). A further test of the inuence of leading indicators on the probability of US business phase shifts. International Journal of Forecasting 14, 6370.