European Journal of Operational Research 148 (2003) 514–524

www.elsevier.com/locate/dsw

Decision Aiding

Analyzing AHP-matrices by regression

Pertti Laininen *, Raimo P. H€
am€
al€
ainen
Systems Analysis Laboratory, Helsinki University of Technology, P.O. Box 1100, Fin-02015 Hut Helsinki, Finland
Received 26 October 2000; accepted 15 April 2002

Abstract

In the analytic hierarchy process (AHP) the decision maker makes comparisons between pairs of attributes or al-
ternatives. In real applications the comparisons are subject to judgmental errors. Many AHP-matrices reported in the
literature are found to be such that the logarithm of the comparison ratio can be sufficiently well modeled by a normal
distribution with a constant variance. On the basis of this model we present the formulae for the evaluation of the
standard deviations of the estimates of the AHP-weights obtained by regression analysis. In order to eliminate the effect
of an outlier in the comparison ratios a robust regression technique is elaborated, and compared with the eigenvector
method and the logarithmic least squares regression. A dissimilarity matrix approach is presented for the statistical
simultaneous comparisons of the AHP-weights. The results are illustrated by simulation experiments.
Ó 2002 Elsevier Science B.V. All rights reserved.

Keywords: Analytical hierarchy process; Eigenvector method; Regression; Robust regression; Multiple comparisons; Simultaneous
comparisons

1. Introduction example, an inconsistency can also be seen as

The standard method to calculate the values for
the weights from an analytic hierarchy process
(AHP)-matrix is to take the eigenvector corre-
sponding to the largest eigenvalue of the matrix,
and then to normalize the sum of the components
equal to one (Saaty, 1977, 1980). A drawback of
this method is that there is no practical statistical
theory behind it. A statistical approach is needed if
one thinks that the ratios of the relative impor-
tance of entities contain random fluctuations. For
*
Corresponding author. Tel.: +358-9-451-3061; fax: +358-9-
451-3096.
E-mail address: pertti.laininen@hut.fi (P. Laininen).
random variation. There are many studies showing
that in human decision making inconsistencies can
be expected. This is widely acknowledged both in
the AHP literature and elsewhere (for instance in
Fischoff et al., 1980). Also, Genest and Rivest
(1994) give a discussion about inconsistency of
comparisons made by experts or decision makers.
The comparison scale also has an effect on the
inconsistencies (P€oyh€onen et al., 1997; Salo and
H€am€al€ainen, 1997). In this paper we do not con-
sider the effects of the comparison scale.
The regression approach as a logarithmic least
square has been proposed by many authors (for
instance De Jong, 1984; Saaty and Vargas, 1984;
Crawford and Williams, 1985; Fichtner, 1986;
Alho and Kangas, 1996). De Jong (1984) gives a

0377-2217/03/$ - see front matter Ó 2002 Elsevier Science B.V. All rights reserved.
doi:10.1016/S0377-2217(02)00430-7
 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524
profound statistical treatment of the use of regres-
sion analysis for analyzing AHP-matrices. In this
approach the ratio rij , the relative value of attribute
i compared to attribute j, given by the decision
maker, is considered a value of a random variable.
The natural logarithms logðrij Þ are regressed with
the dummy variables corresponding to the attri-
butes. De Jong suggests the use of the coefficient
of determination of the regression equation as a
measure of consistency in the decision maker’s re-
sponses. This quantity has a close relationship with
the residual mean square of the regression, which
Crawford and Williams (1985) propose as their
measure of consistency. This method often gives
very similar weights as the eigenvector method.
Genest and Rivest (1994) prove analytically that
the difference between the estimates of the weights
given by the eigenvector method and the regression
analysis is of order Oðr2 Þ, where r2 is the variance of
logðrij Þ. The use of linear models and the experi-
mental design in AHP are discussed in Alho and
Kangas (1996). They present a theory of variance
components model and the least squares estimation
of preferences. In Alho and Kangas (1997) a Bayesian
extension of the regression technique is presented.
The regression approach enables to estimate the
weights and their standard errors. Also, it enables
the use of a robust regression (Montgomery and
Peck, 1992) discussed by Laininen and H€ am€ al€
ainen
(1999). The robust regression is robust for excep-
tional values of rij . Like in the regression analysis
an exceptional value is called an outlier. In general,
statistical regression software are equipped with a
possibility to study if there are outliers. For in-
stance, one can observe the standardized deleted
residuals. Robust regression is a technique that
automatically gives a solution that is not affected
by outliers.
The regression approach enables the statistical
simultaneous comparisons of AHP-weights, too,
which gives a deeper insight into the differences in
the weights.
1.1. Statistical model
In the AHP attributes are compared in a com-
parison matrix, and the decision alternatives are
compared in a comparison matrix with respect to
515
each attribute. These comparisons give local
weights of the attributes and alternatives. The
global weights of the decision alternatives are
calculated by product sums of the local attribute
weights and alternative weights. In this process the
accuracy of the estimates of the local weights is a
matter of vital importance. In this paper we dis-
cuss the analysis of the AHP-matrix that gives the
local weights.
Let us take an AHP-matrix of size m  m with
the entries rij , i; j ¼ 1; . . . ; m. rij is the relative value
of attribute i compared to attribute j as perceived
by the decision maker. The entries rij make the
pairwise comparisons data with the reciprocal re-
lation rji ¼ 1=rij for i; j ¼ 1; . . . ; m.
Let w1 ; . . . ; wm (w1 > 0; . . . ; wm > 0) be the true
weights of the attributes with the condition
w1 þ    þ wm ¼ 1, and let v1 ; . . . ; vm (v1 > 0; . . . ;
vm > 0) be values of the attributes so that the nor-
malized weights w1 ; . . . ; wm can be calculated from
v1 ; . . . ; vm by normalizing the sum equal to one.
Now, the observation rij is an observation from
the ratio vi =vj , and the logarithm logðrij Þ is an ob-
servation from logðvi =vj Þ ¼ logðvi Þ  logðvj Þ. The
analysis begins by modeling the observation error.
Vargas (1982) assumes that the random vari-
ables rij follow a gamma distribution, and then the
joint distribution of the estimates of the weights
w1 ; . . . ; wm is a Dirichlet distribution. The marginal
distributions of the Dirichlet distribution are beta
distributions, a fact that gives an idea to model the
individual distribution of each estimated weight as
a beta distribution. Differently from Vargas we
assume that rij is a logarithmic normally distrib-
uted random variable. The decision maker uses
integers from one to nine in her/his comparisons,
and it is clear that logðrij Þ is not exactly normally
distributed. But, one can find many AHP-matrices
reported in the literature on AHP-applications
(for instance Kangas et al., 1992; Leskinen, 2000)
where the residuals of the logarithmic least square
behave like the residuals of a regression with
normally distributed errors. We will model the
analysis of AHP-matrices by the regression anal-
ysis on the basis of the model
logðrij Þ  N ðlogðvi Þ  logðvj Þ; r2 Þ;
i ¼ 1; . . . ; m  1; j ¼ 2; . . . ; m; ð1Þ
516 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524
where r2 is a constant variance, and the random
variables logðrij Þ can be supposed to be uncorre-
lated. We will introduce this approach in Section
2. This approach is identical with the well-known
model rij ¼ expðlogðvi Þ  logðvj Þ þ ij Þ, where the
random variables ij follow normal distribution.
The standard errors of the weights given by the
regression approach can be calculated from the
solution of the regression analysis by linearizing
the formula of the weights. We will present a way
to calculate the standard errors based on model
(1).
Model (1) is in accordance with the geometric
measurement scale rij ¼ expðsdij Þ, where s > 0 is
a measurement scale parameter, and dij , i; j ¼
1; . . . ; m are values which are dependent on the
decision maker (Lootsma, 1993).
Robust regression is a regression technique that
enables to estimate the weights so that an outlier
will not have an effect on the solution. We will
introduce the idea of the robust regression, and
give illustrations of its use in Section 3.
After regression analysis statistical comparisons
of the weights can be done. Let rij be a comparison
ratio given by the decision maker. One can calcu-
late the corresponding value r^ij given by the re-
gression model for all i 6¼ j. A large difference
between rij and ^rij tells about problems in the
comparison of the attributes i and j. In order to
statistically test the hypothesis H0 : w1 ¼    ¼ wm
for all i 6¼ j we establish a dissimilarity matrix that
includes a test statistic for every difference logðvi Þ 
logðvj Þ. Observe that logðvi Þ  logðvj Þ ¼ 0 ) wi 
wj ¼ 0, and wi  wj ¼ 0 ) logðvi Þ  logðvj Þ ¼ 0.
r ¼ 0:4 which is assumed to be typical in experi-
mental AHP-matrices. For instance, in the data of
Leskinen (2000) the value of r varies from 0.4 to
0.8. The results given by r ¼ 0:4 remain similar for
the value r ¼ 0:8, too.
Efron and Tibshirani (1993) present studies on
the number of simulations needed in different sit-
uations. In the estimation of a standard error 200
simulations are generally enough. Here we have
used 2000 simulations. In the estimation of the
quantile of a small tail probability we have made
40 000 simulations.
2. Standard errors of the estimates of the weights
Let us take an AHP-matrix of size m  m with
the entries rij , i, j ¼ 1, . . ., m comprising the pair-
wise comparisons data. We first convert the data
into a linear regression form by a logarithmic
transformation.
We write
logðrij Þ ¼ yij ; logðvi Þ  logðvj Þ ¼ bi  bj ;
i; j ¼ 1; . . . ; m: ð2Þ
According to the model (1), for the total of
mðm  1Þ=2 comparisons the following equations
can be written:
yij ¼ bi  bj þ ij ; i ¼ 1; . . . ; m  1;
j ¼ 2; . . . ; m; ð3Þ
where ij , the residuals, are uncorrelated random
variables which are normally distributed with ex-
The significances of the test statistics are evaluated pectation E½ij  ¼ 0, and with constant variance
simultaneously. This will be presented in Section 4. Var½ij  ¼ r2 . Let y be the mðm  1Þ=2  1 vector
of yij , b the m  1 vector of bi ,  the mðm  1Þ=
1.2. Simulation experiments 2  1 vector of ij , and X the mðm  1Þ=2  m
matrix of the dummy variables given by the system
We have illustrated the questions by simulation of Eq. (3). The equations can be written
experiments. In the simulation experiment AHP-
matrices are generated by using the model (1). An y ¼ Xb þ : ð4Þ
entry rij is obtained by adding into the expectation
a normally distributed random error with the The relationship between the weight wi and the
standard deviation r, and then rounding its ex- parameters bj is
ponential to the nearest fraction formed by inte-
gers from one to nine. In general, we present expðbi Þ
wi ¼ ; i ¼ 1; . . . ; m: ð5Þ
simulation results calculated by using the value expðb1 Þ þ    þ expðbm Þ
 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524 517

The estimates for the parameters bi are then
calculated by minimizing
P some function d of the
residuals ij , dðij Þ. If dðij Þ ¼ 2ij , the least
squares regression is done. The solution needs an
ance s2i that is estimated by the regression analysis.
Then expðb^i Þ for i ¼ 1; . . . ; m  1 follow a log-
normal distribution with expectation
E½expðb^i Þ ¼ expðbi þ s2i =2Þ; ð7Þ
additional constraint for the parameters because
the model (4) is over-parametrized. In fact, only and variance
the differences bi  bj can be estimated from the  
comparison matrix. These differences are sufficient
to determine the weights wi . Take for instance
bi  bj ¼ dij , or bi ¼ bj þ dij . Then
expðbj þ dij Þ
wi ¼
expðbj þ d1j Þ þ    þ expðbj þ dmj Þ
Var½expðb^i Þ ¼ expð2bi þ s2i Þ expðs2i Þ  1 : ð8Þ
See for instance Ghahramani (1996). Because of
the sum b^i þ b^j being normally distributed (see
Rao, 1968) the expectation of the product
expðb^i Þ expðb^j Þ can be calculated according to
Eq. (7), and the variance–covariance matrix
expðdij Þ Cov½expðb^i Þ; expðb^j Þ will have the entries
¼ : ð6Þ
expðd1j Þ þ    þ expðdmj Þ
Cov½expðb^i Þ; expðb^j Þ ¼ expðbi þ bj þ ðs2i þ s2j Þ=2Þ
Thus, the value for bj can be chosen arbitrarily.  ðexpðCov½b^i ; b^j Þ  1Þ
A practical constraint is bm ¼ 0 (or vm ¼ 1). Then
the last column of the matrix X can be rejected, ð9Þ
and the new matrix X is of size mðm  1Þ= for i 6¼ j. Since b^m ¼ 0, a constant, E½expðb^m Þ ¼ 1,
2  ðm  1Þ, and the vector b is of size ðm  1Þ  1,
T Var½expðb^m Þ ¼ 0, and Cov½expðb^i Þ; expðb^m Þ ¼ 0
b ¼ ðb1 ; . . . ; bm1 Þ . The method gives a unique
for i ¼ 1; . . . ; m  1.
^
solution b1 ; . . . ; bm1 with b^m ¼ 0. In the least
^
The estimates for the covariances are calculated
squares regression the estimates b^1 ; . . . ; b^m1 are d b^i ; b^j 
by inserting the estimates b^i , s^2i , and Cov Cov½
unbiased and normally distributed, and the co-
into formula (9). The estimate of the variance s^2i
variance matrix Cov½b^i ; b^j  is a standard result of d b^i ; b^i . The inserting estimates are
is equal to Cov Cov½
the regression analysis.
results of the regression analysis.
The regression analysis gives b^i estimates,
^ By forming the derivatives of wi in terms of
bm ¼ 0, and the estimates of wi weights are ob-
expðbi Þ in formula (5) the covariance matrix
tained by inserting into formula (5).
Cov½w ^ j  can be written as
^i ; w
 
owi
2.1. Standard errors of the estimates Cov½w ^j  ¼
^i ; w
oebj mðm1Þ
The usual way to evaluate the variances of the  Cov½expðb^i Þ; expðb^j Þðm1Þðm1Þ
estimates w ^ i is to linearize formula (5) in terms of  T
owi
the parameters bi , and then evaluate the variance  : ð10Þ
of the linear expression (see for instance Alho and oebj ðm1Þm
Kangas, 1996). But formula (5) is strongly non- Here we have
linear with respect to the bi , especially for positive 2 3
w1  1 w1  w1
values of bi , and the linearization may give very 6 w2 7
  6 w2  1  w2 7
incorrect values for the variances. Better estimates owi 6 7
¼ wm 6         7:
for the variances can be obtained by calculating oebj 6 7
mðm1Þ 4 wm1 wm1  wm1  1 5
the variances and covariances of the functions
wm wm  wm
expðb^i Þ, i ¼ 1; . . . ; m theoretically which is possible
under the model (1). Then we need to linearize the ð11Þ
formula (5) in terms of expðbi Þ only. The estimate The estimates of the covariances are calculated
b^i is unbiased and normally distributed with vari- by inserting the estimates of the weights into
518 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524

(11) and the estimate of the covariance matrix vation vector of the response Y in n trials, X is
Cov½expðb^i Þ; expðb^j Þ into (10). the design matrix of size n  p of the p regressors
T
The standard error Se½w ^ i  is then calculated as a
square root of the diagonal element ði; iÞ of the
matrix (10).
2.2. Simulation of the standard error
By simulation experiments we can see that the
X1 ; . . . ; Xp , b ¼ ðb1 ; . . . ; b p Þ is the parameter vec-
T
tor, and  ¼ ð1 ; . . . ; n Þ is a random vector of
independent random errors i , residuals with ex-
pectation zero. Let xi ¼ ðxi1 ; . . . ; xip Þ be the ith row
of the matrix X.
The general approach to the estimation of the
parameters b is to minimize a function d of the
standard errors Se½w ^ i  given by (10) and (11) are residuals,
practically unbiased. We generated 2000 5  5
AHP-matrices from the matrix with equal weights X
n X
n

by using r ¼ 0:4, and calculated the standard de- min dði Þ ¼ min dðyi  xi bÞ: ð12Þ
b b
i¼1 i¼1
viations of the 2000 values given by the eigenvector
method and given by the logarithmic least squares
regression. The values for the standard deviations An estimate of this type is often called an M-
were for each of the five weights and for both of the estimate, where M stands for maximum likelihood.
estimation methods 0.032. In each simulation the There are many possibilities for the form of the
standard errors of the weight estimates, Se½w ^ i , function d which is usually treated as a function
were calculated both using formulae (10) and (11) of the stadardized residual z. If we take dðzÞ ¼
and the method of linearizing the formula of wi z2 =2, the regression is the usual least squares re-
with respect to bj parameters. The mean of the gression. The function dðzÞ ¼ jzj gives the so called
standard errors Se½w^ i  was 0.032 for each weight wi .
The mean values are equal to the values of the
corresponding standard deviations. Alho and
Kangas (1997) note that the values of the standard
errors calculated by linearizing with respect to bj
parameters are often too small. In our simulations
the values were in all cases less than the values given
by (10) and (11) deviating 40% as its maximum.
Simulations based on matrices with unequal
weights give results which are consistent with the
above results.
When using the standard errors of the weights in
practice one should notice that the sampling dis-
tributions of the estimates of the weights having
values near zero or near one are heavily skew.
The sampling distributions given by our simula-
tions resembled the beta distribution proposed by
Vargas (1982).
3. Robust regression analysis and AHP
The theory of robust regression can be found in
standard statistical texts (see for instance Mont-
gomery and Peck, 1992).
We take a regression model in general form (4)
T
y ¼ Xb þ , where y ¼ ðy1 ; . . . ; yn Þ is the obser-
absolute deviation regression.
The estimates of the parameters are calculated
by deriving the Eq. (12) with respect to each bi and
putting the derivatives equal to zero. It is known
that in the case of dðzÞ ¼ z2 =2 the equation system
resulting from the derivation is a linear one, the
least squares normal equations. If the function
dðzÞ is of some other type, then the equation sys-
tem is not linear. However, the equations can still
be approximated by the equations of a weighted
linear regression, and the values for the esti-
mates of bi can be solved by iteration by carrying
out weighted linear regressions. The weights are
functions of the residuals, and their functional
form is
d 0 ðzÞ
gðzÞ ¼ : ð13Þ
z
For the least squares regression the weight
function is gðzÞ ¼ 1. The idea in the robust re-
gression is to give a weight equal to one if the
absolute value of the residual z is small. In the case
of a large standardized residual, for instance an
absolute value greater than two, the value of the
weight function should be smaller than one. If the
residual is large enough, an outlier, the weight
 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524
should be zero. In analyzing AHP-matrices we
have tried to find such a weight function that the
robust regression works like the logarithmic least
squares regression if there is only normal random
variation, but if there is an outlier, then the robust
regression gives a solution which is not strongly
affected by this. The reader should avoid confu-
sions and note that the regression weight functions
are not the same as the AHP-weights. We suggest
the following weight function for the robust re-
gression:
 This method is known as the iteratively reweighted
1 if jzj 6 b
gðzÞ ¼ ð14Þ
expðaðjzj  bÞÞ if jzj > b:
Here a and b are positive constants. If we have
the values for the residuals zi , the corresponding
values for the weights gðzi Þ, i ¼ 1; . . . ; n can be
calculated.
The statistical computer programs in general
contain a module for robust regression. However,
one should note that the programs do not neces-
sarily use exactly same algorithms as we have used.
In the following we give all formulae for calcula-
tions of our results.
Let G be a n  n matrix with entries
Gði; iÞ ¼ gðzi Þ, i ¼ 1; . . . ; n, and Gði; jÞ ¼ 0 for all
i 6¼ j ¼ 1; . . . ; n. Then the estimates b for the pa-
rameters b are
1
b ¼ ðXT GXÞ XT Gy:
Let MSE be the mean square
T same weight as in the least squares regression. If
MSE ¼ ðy  XbÞ Gðy  XbÞ=ðn  pÞ;
and H the hat matrix
1
H ¼ XðXT GXÞ XT G:
The standardization of the residual yi  xi b is
done by dividing it with the standard error
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
519
we have m AHP-weights and p ¼ m  1 regressors
in the regression analysis.
The estimates for the bi are iterated by taking at
the first step Gði; iÞ ¼ 1, i ¼ 1; . . . ; n, and calcu-
lating the first estimates. At the second step the
weights Gði; iÞ are calculated by using the weight
function with the known residuals from the first
step. At the third step the residuals given by the
second step are used, and so on. After some iter-
ations the values of the estimates do not change,
and the final results are taken from the last step.
least squares regression (Montgomery and Peck,
1992).
There are some differences between the dis-
tributions of the estimates given by the usual
weighted regression and this iteratively reweighted
regression. The estimates (15) of the regression
coefficients are normally distributed asymptoti-
cally only, and the variance covariance matrix (19)
is an approximation. One can raise the question
concerning the normalization of the residuals. We
have used the standard error (18) at each step of
iteration, but there are also other kinds of pro-
posals (see Montgomery and Peck, 1992). This
is one reason for differences between statistical
computer programs.
3.1. Choosing the values for a and b
ð15Þ
If for a residual jzj 6 b, then the residual has the
ð16Þ
b ¼ 2, then about 95% of the residuals have the
weight of the least squares regression. For residu-
als jzj > b the weight is determined by the pa-
ð17Þ
rameter a. The greater the value a > 0 is the more
rapidly the weight of the residual diminishes to
zero.
We have studied the robust regression by sim-
Se½i  ¼ MSEð1  H ði; iÞÞ; ð18Þ
where H ði; iÞ is the entry ði; iÞ of H. The variance
covariance matrix of the estimates b ¼ b^ is
ulation experiments. The idea was to seek such a
weight function that robust regression gives results
which are consistent with the results given by the
logarithmic least squares regression, if there is
Cov½b ¼ ðXT GXÞ1 MSE:
The results in (15)–(19) are standard results
from regression analysis (see for instance Mont-
gomery and Peck, 1992). In our AHP-application
ð19Þ
usual random variation only, and that an outlier
has a minor effect on the weights given by the
robust regression. As we can see the weight func-
tion (14) with a ¼ 1 and b ¼ 2 works well for this
520 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524

purpose. At the same time we compare the results
with the eigenvector method, too.
Let us take an AHP-matrix A of size 5  5:
2 3
1 2 5 5 9 Table 1. The effect of the values of a and b is such
6 1=2 1 5=2 5=2 9=2 7
6 7
A¼6
6 1=5 2=5 1 1 9=5 7
7:
4 1=5 2=5 1 1 9=5 5
1=9 2=9 5=9 5=9 1
The matrix A is fully consistent so that the
consistency ratio (Saaty, 1980) takes the value
CR ¼ 0. Both the eigenvector method, the loga-
rithmic least squares regression, and the robust re-
gression give the same weights: w1 ¼ 0:4972, w2 ¼
0:2486, w3 ¼ 0:0994, w4 ¼ 0:0994, w5 ¼ 0:0552. We
call these the correct weights.
In order to compare the eigenvector method, the
logarithmic least squares regression, and the robust
regression we generated 2000 random matrices
from A by adding into each entry logðAði; jÞÞ a
normally distributed random error with expecta-
tion zero and standard deviation 0.4, and rounding
its exponential to the nearest fraction formed by
the integers from one to nine. The value 0.4 for the
standard deviation represents a typical one in ex-
perimental data of Leskinen (2000). For each ma-
trix the weights given by the eigenvector method,
the least squares regression, and the robust re-
gression were calculated. The means and the stan-
dard deviations are shown in the following table.
One can see from Table 1 that the values of the
means and the values of the standard deviations
given by the logarithmic least squares regression
and the robust regression are practically identical,
and the means are near to the correct weights. The
means given by the eigenvector method indicate a
small bias.
Our simulations by using the value r ¼ 0:8 give
results which are consistent with the results in
that, if we choose b < 2 or a > 1 the standard
ð20Þ deviations given by the robust regression become
larger.
3.2. The effect of an outlier
In the following comparisons we demonstrate
how the eigenvector method, the ordinary least
squares regression, and the robust regression work
in the presence of an outlier. We have changed the
value of an entry of the consistent matrix A used
before. The values of the weights can be seen in the
last column in Table 1.
The correct value of the entry Að1; 4Þ is 5, and we
changed it to 7 together with Að4; 1Þ ¼ 1=7. The
resulting consistency ratio is CR ¼ 0:3. This modi-
fication does not have an effect on the weights given
by the robust regression but the weights given by
the eigenvector method change: w1 ¼ 0:5183, w2 ¼
0:2411, w3 ¼ 0:0964, w4 ¼ 0:0905, w5 ¼ 0:0536.
Also the corresponding weights given by the loga-
rithmic least squares regression change: w1 ¼
0:5173, w2 ¼ 0:2418, w3 ¼ 0:0967, w4 ¼ 0:0904,
w5 ¼ 0:0537. If we modify the value of the entry
Að1; 4Þ a little more so that Að1; 4Þ ¼ 9 and
Að4; 1Þ ¼ 1=9, then CR ¼ 0:9. The weights given by
the robust regression are exactly the same as before
the modification, but the weights given by the two
other methods change radically. The eigenvec-
tor method gives the weights w1 ¼ 0:5356, w2 ¼
0:2343, w3 ¼ 0:0937, w4 ¼ 0:0843, and w5 ¼ 0:0521.

Table 1
The means and the standard deviations of the weights from 2000 simulations
Weight Mean eigenv. StDev eigenv. Mean log.ls. StDev log.ls. Mean rob.reg. StDev rob.reg. Correct weight
w1 0.482 0.048 0.493 0.052 0.493 0.054 0.4972
w2 0.254 0.040 0.250 0.041 0.250 0.042 0.2486
w3 0.103 0.019 0.101 0.019 0.101 0.020 0.0994
w4 0.102 0.020 0.100 0.020 0.100 0.020 0.0994
w5 0.059 0.010 0.056 0.011 0.056 0.011 0.0552
Eigenv: ¼ weights calculated by the eigenvector method. Log:ls: ¼ weights calculated by the logarithmic least squares regression.
Rob:reg: ¼ weights calculated by the robust regression with a ¼ 1 and b ¼ 2.
 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524 521

The logarithmic least squares regression gives the
values w1 ¼ 0:5321, w2 ¼ 0:2366, w3 ¼ 0:0946,
w4 ¼ 0:0841, and w5 ¼ 0:0526.
As these examples show the eigenvector method
and the logarithmic least squares regression are
quite sensitive to outliers. The robust regression
least once in the paired tests, then H0 of the
composite hypothesis is rejected. The paired test
H0 : bi ¼ bj can be carried out by using the test
statistic
b^i  b^j
tij ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ;
automatically reduces the effect of an outlier. d b^i ; b^i  þ Cov
Cov½
Cov d b^j ; b^j   2 Cov
4. Statistical simultaneous comparison of the
weights
As a result of the regression analysis one ob-
taines the estimates for the AHP-weights and the
standard errors of the estimates, too. But there are
other features in the regression analysis which help
the analyst interpret the results.
The estimated matrix A ^ contains the values of
the entries given by the regression model:
A^ði; jÞ ¼ expðb^i  b^j Þ; i; j ¼ 1; . . . ; m:
A comparison between the entries Aði; jÞ given
by the decision maker and the estimated entries
A^ði; jÞ gives an insight into the inconsistency of the
statements given by the decision maker. A high
difference between the observed and the estimated
values tells about the inconsistency of the decision
maker’s statements.
By using the results of the regression analysis a
multiple comparison of the weights can be done.
This means testing the hypothesis H0 : w1 ¼    ¼
wm at a risk level a. The easy way to make the test is
to test the hypothesis H0 : b1 ¼    ¼ bm . Namely,
if wi ¼ wj then bi ¼ bj , and vice versa. For testing
the hypothesis concerning the b parameters we
have the theory of linear models in use (see for
instance Miller, 1981).
The simultaneous hypothesis H0 : b1 ¼    ¼ bm
can be tested by testing all of the hypotheses
H0 : bi ¼ bj , i 6¼ j ¼ 1; . . . ; m. If H0 is rejected at
Table 2
Critical values tc ð0:05Þ of tij at the risk level a ¼ 0:05
m 3 4 5
tc ð0:05Þ 18.94 4.79 3.76
The values represent 95% quantiles of 40 000 simulated values for the distribution of jtij j with equal correct weights.
Cov½ d b^i ; b^j 
Cov½
i < j ¼ 1; . . . ; m: ð22Þ
The denominator in (22) is the standard error of
the difference b^i  b^j . For i ¼ j we put tii ¼ 0. From
a consistent comparison matrix the w ^ i estimates are
obtained so that the standard errors of the esti-
mates are zero. In this case tij takes the value in-
finity for i 6¼ j if wi  wj 6¼ 0. All values tij ,
i; j ¼ 1; . . . ; m establish an m  m matrix T that we
call the dissimilarity matrix. If the model (1) is valid
and bi  bj ¼ 0 (or wi  wj ¼ 0), then according
to standard regression theory the standardized
difference tij follows the t-distribution with a
ð21Þ
df ¼ n  m þ 1 degrees of freedom. So, matrix T
provides a simple way to the statistical simulta-
neous comparison of the AHP-weights in order to
decide which of them differ statistically signifi-
cantly. The critical value tc ðaÞ for the statistic tij is
the value that maxðjtij jÞ exceeds with the proba-
bility a. In the theory of linear models the critical
value is usually given by the assumption that the
estimates b^i and b^j are independent, and the critical
value is then obtained from the Studentized range
distribution. But here the estimates are correlated,
and so the critical values must be simulated. In
Table 2 we give simulated critical values for m ¼
3; . . . ; 10 and a ¼ 0.05. If some of the values of jtij j
in the matrix T exceeds the corresponding critical
value in Table 2, then the hypothesis H0 : w1 ¼
   ¼ wm is rejected at the risk level 0.05.
A fine feature of the test statistic (22) is that it is
independent on the scale parameter of the Loot-
sma’s exponential scale (Leskinen, 2000).
6 7 8 9 10
3.46 3.39 3.34 3.36 3.37
522 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524

4.1. Grouping of the weights Weight w1 w2 w3 w4 w5 w6

^i
w 0.316 0.139 0.036 0.125 0.236 0.148
If the hypothesis H0 : w1 ¼    ¼ wm is rejected Se ½w
^i  0.110 0.061 0.018 0.056 0.093 0.062
the question arises how the weights can be grouped
into homogenous subgroups. The weights wi and
wj belongs to the same subgroup if they do not The values of the standard error are dependent
differ significantly from each other, but in the case on the values of the weights. A large weight has a
of a significant difference they are grouped into large standard error, and a small weight has a
different subgroups. There are several grouping small standard error. Robust regression gives here
methods in the theory of linear models (see for the same values for the weights as the logarithmic
instance Miller, 1981). Here our problem is not least squares regression: w ^ 1 ¼ 0:316, w
^ 2 ¼ 0:139,
a really statistical testing problem. Therefore, the w^ 3 ¼ 0:036, w^ 4 ¼ 0:125, w
^ 5 ¼ 0:236, w^ 6 ¼ 0:148.
grouping method does not need to handle the error The estimates calculated by the eigenvector method
rates as exactly as possible. The method presented differ slightly from these values: w ^ 1 ¼ 0:321,
by Fisher (see Miller, 1981) is suitable. By this w^ 2 ¼ 0:140, w^ 3 ¼ 0:035, w
^ 4 ¼ 0:128, w^ 5 ¼ 0:237,
method the weights wi and wj are grouped into the w^ 6 ¼ 0:139. Because there is no difference between
same subgroup if jtij j does not exceed the usual the logarithmic least squares and the robust re-
critical value in the t-test. This critical value can be gression there is no problem with the outliers.
found from the table of the t-distribution. The dissimilarity matrix T is the following:
If the decision maker has a clear opinion of the 2 3
priorities of the attributes under comparison then 0 1:60 4:23 1:80 0:57 1:48
6 1:60 0 2:63 0:21 1:03 0:12 7
the discovered subgroups are disjoint. In a more 6 7
6 4:23 2:63 0 2:42 3:66 2:75 7
uncertain case some of the weights may belong to T¼6
6 1:80 0:21
7:
6 2:42 0 1:24 0:32 7
7
two subgroups. 4 0:57 1:03 3:66 1:24 0 0:91 5
Example: In the following we analyze an actual
1:48 0:12 2:75 0:32 0:91 0
AHP-matrix given by Saaty (2000, p. 113). In his
example Saaty asked his son to evaluate three high In the test of H0 : w1 ¼    ¼ w6 the critical va-
schools according to six criteria with the charac- lue for tij is tc ð0:05Þ ¼ 3.46 in Table 2. As we can
teristics: learning, friends, school life, vocational see from the matrix T there are two values greater
training, college preparation, and music classes. than 3.46. Thus the hypothesis H0 is rejected at the
He also weighted the relative importance of these risk level 0.05. Here the decision maker, indeed,
criteria with regard to the satisfaction with the has an opinion that all the characteristics are not
school. The comparison ratios of the criteria in the equally important. After this result grouping into
above mentioned order are given in the following subgroups is allowed.
matrix. The test statistic tij has 10 degrees of freedom,
2 3 and the corresponding t-value is tð0:05Þ ¼ 2:228
1 4 3 1 3 4 (see the table of t-distribution). From the dissimi-
6 1=4 1 7 3 1=5 1 7 larity matrix T we can see that all values jt3j j,
6 7
6 7 j ¼ 1; 2; 4; 5; 6 are greater than 2.228, and these are
6 1=3 1=7 1 1=5 1=5 1=6 7
A¼6
6 1
7: the only values greater than 2.228. The weights, or
6 1=3 5 1 1 1=3 7
7 characteristics, can be grouped into two disjoint
6 7
4 1=3 5 5 1 1 3 5 groups: fw3 g, and fw1 ; w2 ; w4 ; w5 ; w6 g.
1=4 1 6 3 1=3 1 The statistical conclusion is the following: For
this decision maker the criterion school life is the
The logarithmic least squares regression gives the only one having the different weight than the other
following estimates and the standard errors for criteria. It is smaller than the other weights among
the weights w1 ; . . . ; w6 of the characteristics learn- which he does not have differences compared to
ing, . . ., music classes: each other.
 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524
5. Conclusions
In AHP the ratio statements about the relative
importance of the weights given by a decision
maker can be thought of to include random vari-
ations. We have presented a new analysis method
for AHP-matrices which is based on the hypothe-
sis that the logarithm of the comparisons ratio
follows a normal distribution with a constant
variance. The method includes the use of robust
regression together with the eigenvector method
or the logarithmic least squares regression, and
the statistical multiple comparison of the weights
by testing the hypothesis H0 : w1 ¼    ¼ wm and
grouping the weights.
In the robust regression we have developed such
a weighting function of the residuals that the ro-
bust regression and the logarithmic least squares
regression give similar values for the weight esti-
mates and their standard errors if there is normal
random variation only. The presence of an outlier
does not have a significant effect on the estimates
given by the robust regression. In practice, if the
weight estimates calculated by the eigenvector
method or by the logarithmic least squares re-
gression are near the values given by the robust
regression, then there do not exist any outlier
problems. But if the results given by the robust
regression differ considerably from the results of
the standard methods, then the comparison matrix
should be reassessed. If there is an actual error in
the comparison ratios, the error must be corrected.
A rough comparison of the weight estimates
calculated by different methods can be done by
studying the differences of the estimates with re-
spect to the standard errors of the weights given by
the robust regression. We have presented how the
standard errors of the weight estimates can be
calculated in the case that the comparison ratios
follow a lognormal distribution.
In the simultaneous comparison of the weights
we test the hypothesis H0 : w1 ¼    ¼ wm by test-
ing H0 : b1 ¼    ¼ bm , where bi ¼ logðvi ), and vi is
the value of the attribute or alternative i. The last
modification of the test allows the use of the theory
of linear models. The test is done by studying the
dissimilarity matrix formed by the t-statistics for
the paired tests H0 : bi ¼ bj , i 6¼ j. The critical
523
values for the dissimilarity matrix at the risk level
0.05 are calculated by simulations, and tabulated.
If the hypothesis H0 is rejected the weights are
grouped into homogenous subgroups by using the
Fisher’s approach. The purpose of the simulta-
neous comparison is to evaluate to what extent the
decision maker is able to find differences in the
importances of the attributes or alternatives, and
to what extent his/her comparisons are random
entries only. If there are significant differences
between the weights, and if the grouping leads into
two or more disjoint subgroups, then the decision
maker has succeeded in differentiating the priori-
ties.
Of course, in some special cases the decision
maker may have the opinion that all the attributes
or alternatives are equally important.
References
Alho, J.M., Kangas, J., 1997. Analyzing uncertainties in
experts’ opinions of forest plan performance. Forest Science
43, 521–528.
Alho, J.M., Kangas, J., Kolehmainen, O., 1996. Uncertainty in
expert predictions of the ecological consequences of forest
plans. Appl. Stat. 45, 1–14.
Crawford, G., Williams, C., 1985. A note on the analysis of
subjective judgement matrices. Journal of Mathematical
Psychology 29, 387–405.
De Jong, P., 1984. A statistical approach to Saaty’s scaling
method for priorities. Journal of Mathematical Psychology
28, 467–478.
Efron, B., Tibshirani, R.J., 1993. An Introduction to the
Bootstrap. Chapman & Hall, New York.
Fichtner, J., 1986. On deriving priority vectors from matrices of
pairwise comparisons. Socio-Economic Planning Sciences
20, 399–405.
Fischoff, B., Slovic, B., Lichtenstein, S., 1980. Knowing what
you want: measuring labile values. In: Wallsten, T.S. (Ed.),
Cognitive processes in choice and decision behavior. Law-
rence Erlbaum Associates, Hillsdale, pp. 285.
Genest, C., Rivest, L.-P., 1994. A statistical look at Saaty’s
method of estimating pairwise preferences expressed on a ratio
scale. Journal of Mathematical Psychology 38, 477–496.
Ghahramani, S., 1996. Fundamentals of Probability. Prentice-
Hall, New Jersey.
Kangas, J., Matero, J., Pukkala, T., 1992. Analyyttisen
hierarkiaprosessin k€aytt€ o metsien monik€ayt€ on suunni-
ttelussa-tapaustutkimus (Using the analytic hierarchy pro-
cess in planning of multiple-use forestry––a case study).
Finnish Forest Research Institute Research Notes 412, 48,
in Finnish.
524 P. Laininen, R.P. H€am€al€ainen / European Journal of Operational Research 148 (2003) 514–524
Laininen, P., H€
am€al€
ainen, R.P., 1999. Analyzing AHP-matrices
by robust regression. In: Proceedings of the Fifth Interna-
tional Symposium on the Analytic Hierarchy Process
(ISAHP’99), Kobe, Japan, August 12–14.
Leskinen, P., 2000. Measurement scales and scale independence
in the analytic hierarchy process. Journal of Multi-criteria
Decision Analysis 9, 163–174.
Lootsma, F.A., 1993. Scale sensitivity in the multiplicative
AHP and SMART. Journal of Multi-criteria Decision
Analysis 2, 87–110.
Miller Jr., R.G., 1981. Simultaneous Statistical Inference,
second ed. Springer Verlag, New York.
Montgomery, D.C., Peck, E.A., 1992. Introduction to Linear
Regression Analysis. John Wiley, New York.
P€
oyh€onen, M.A., H€ am€al€ainen, R.P., Salo, A.A., 1997.
An experiment on the numerical modelling of verbal ratio
statements. Journal of Multi-criteria Decision Analysis 6,
1–10.
Rao, C.R., 1968. Linear Statistical Inference and Its Applica-
tions. John Wiley, New York.
Saaty, T.L., 1977. A scaling method for priorities in hierarchical
structures. Journal of Mathematical Psychology 15, 234–
281.
Saaty, T.L., 1980. The Analytic Hierarchy Process. McGraw-
Hill, New York.
Saaty, T.L., 2000. Fundamentals of Decision Making and
Priority Theory with The Analytic Hierarchy Process. RWS
Publications, Pittsburgh.
Saaty, T.L., Vargas, F., 1984. Comparison of eigenvalue,
logarithmic least squares and least squares methods in
estimating ratios. Mathematical Modelling 5, 309–324.
Salo, A.A., H€am€al€ainen, R.P., 1997. On the measurement of
preferences in the analytic hierarchy process. Journal of
Multi-criteria Decision Analysis 6/6, 309–319.
Vargas, L.G., 1982. Reciprocal matrices with random coeffi-
cients. Mathematical Modelling 3, 69–81.