Mathematical and Computer Modelling 46 (2007) 10911098 www.elsevier.

com/locate/mcm

Measuring the efciency of production units by AHP models

Josef Jablonsky
University of Economics, Department of Econometrics, W. Churchill Sq. 4, 130 67 Praha, Czech Republic Received 28 November 2006; accepted 14 March 2007

Abstract The paper deals with models and methods for the evaluation of efciency of production units. The standard modeling approach for the evaluation of efciency is data envelopment analysis (DEA), based on the denition of efciency as the ratio of outputs produced by the unit and inputs spent in the production process. Standard data envelopment analysis models divide the units into inefcient and efcient ones. The efcient units receive an efciency score of 100% with standard DEA models, and can be further classied by so called super-efciency DEA models. This paper discusses the possibility of using an AHP model with interval pairwise comparisons for the evaluation and classication of efcient units, and compares the results with super-efciency DEA scores. The proposed approach is applied to assess the efciency of pension funds in the Czech Republic the results from super-efciency DEA models and from an interval AHP model are compared and discussed. c 2007 Elsevier Ltd. All rights reserved.
Keywords: AHP; Data envelopment analysis; Multiple criteria decision making; Efciency

1. Introduction The importance of pension funds is very high in the current economic conditions in the Czech Republic. This is caused by several factors. The not very positive demographic outlook for the Czech population is one of the most signicant ones. Quite low birth rates, together with longer average life spans in the population, have had a negative impact on the pension system. In this situation, the responsibility of each individual for his or her own retirement income is increasing, and the state supports individuals investing into pension funds. Since 1998, altogether 12 pension funds have been established. All of them offer standard long-term investments intended to bring an above-average rate of return compared to investment instruments that are lower risk. Investment in a pension fund is characterized by the following: it is a saving instrument with a very long time horizon, based on the monthly saving of very low amounts that can lead to a signicant target amount when the pension age is reached, without investing in pension funds, the living standards of most individuals might be signicantly lower after they reach the pension age,

E-mail address: jablon@vse.cz. URL: http://nb.vse.cz/jablon/. 0895-7177/$ - see front matter c 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.mcm.2007.03.007

1092

J. Jablonsky / Mathematical and Computer Modelling 46 (2007) 10911098

the higher rate of return combined with xed monthly investments added to it by the state makes investing in pension funds one of the most economical, and at the same time safest, investment opportunities. Currently, there are 12 pension funds afliated with and controlled by the state that operate in the Czech Republic. The aim of this paper is to analyze the current situation and estimate the efciency of pension funds in order to offer people a recommendation for their investments. The standard modeling tool for the evaluation of efciency is data envelopment analysis (DEA). Standard DEA models categorize evaluated units as efcient or inefcient, but they are not able to further differentiate among the efcient ones. The classication of efcient units can be performed by so-called super-efciency DEA models. Our aim is to develop an AHP model for evaluating efcient pension funds (generally in terms of production units) by means of a DEA model used in the rst stage of the analysis in combination with results from a super-efcient DEA model, leading to the best choice as determined by the AHP model. The paper is organized as follows. The next section contains a brief description of basic DEA models, including the super-efciency models used in our analysis. Section 3 is a brief introduction to AHP models with interval comparisons. Section 4 formulates an IAHP (Interval AHP) model for the evaluation of the efciency of decision making units, and discusses numerical results of both DEA and IAHP analyses. The last section contains a summary of the paper and a discussion about future research. 2. Data envelopment analysis models Data envelopment analysis (DEA) is a tool for measuring and comparing the relative efciency of decision making units (DMU). The DMUs are usually described by several inputs spent to produce several outputs. Let us consider the set E of n decision making units E = {DMU1 , DMU2 , . . . , DMUn }. Each of the units produces r outputs, and for their production m inputs are spent. Let us denote x j = {xi j , i = 1, 2, . . . , m } the vector of inputs and y j = { yi j , i = 1, 2, . . . , r } the vector of outputs of the DMU j . Then X is the (m , n ) matrix of inputs and Y the (r , n ) matrix of outputs. The basic principle of the DEA in the evaluation of efciency of the DMUq , q {1, 2, . . . , n }, consists in looking for a virtual unit with inputs and outputs dened as the weighted sum of inputs and outputs of the other units in the decision set X and Y, where = (1 , 2 , . . . , n ), > 0 is the vector of weights of the DMUs. The virtual unit should be better (or at least not worse) than the analyzed unit DMUq . The problem of looking for a virtual unit can be generally formulated as a standard linear programming problem: minimize z = (eT s+ + eT s ), subject to Y s+ = yq , X + s = xq , , s+ , s 0,

(1)

where eT = (1, 1, . . . , 1) and is a innitesimal constant (usually 108 ). The variables s+ , s are just slack variables expressing the difference between virtual inputs/outputs and appropriate inputs/outputs of the DMUq . Obviously, the virtual inputs/outputs can be computed using the optimal values of variables of the model (1) as follows: xq = xq s , yq = yq + s+ . The DMUq can be considered to be efcient if the virtual unit is identical with the evaluated unit (and there does not exist a virtual unit with better inputs and outputs). In this case, Y = yq , X = xq , and the minimum value of the objective function z = 1. Otherwise, the DMUq is not efcient, and the minimum value of < 1 can be interpreted as the need to proportionally reduce inputs in order to reach the efcient frontier. The model presented here is an input-oriented model, because its objective is to nd a reduction rate of inputs necessary in order to reach efciency. The output-oriented models maximize the expansion rates of outputs in order to reach the efcient frontier. The mathematical formulation of the output-oriented model can be computed as follows: maximize z = + (eT s+ + eT s ), subject to Y s+ = yq , X + s = xq , , s+ , s 0.

(2)

J. Jablonsky / Mathematical and Computer Modelling 46 (2007) 10911098

1093

In model (2), the evaluated unit DMUq is efcient if the optimal objective value z = 1, i.e. = 1, and all the slack variables are equal to zero. The optimal objective function for inefcient units is greater than 1. Models (1) and (2) assume constant returns to scale where a percentage change of inputs leads to the same percentage change of outputs. The modication of the above models that takes into account variable returns to scale is derived from them by adding the convexity constraint eT = 1. The efciency score in standard DEA models is limited to the unit (100%). Nevertheless, the number of efcient units identied by DEA models and reaching the maximum efciency score 100% can be relatively high, and especially in problems with a small number of decision units the efcient set can contain almost all the units. In such cases, it is very important to have a tool for the differentiation and classication of efcient units. That is why several DEA models for the classication of efcient units were formulated. In these models, the efcient scores of inefcient units remain lower than 100% but the the efcient score for efcient units can be higher than 100%. Thus, the efciency score can be taken as a basis for a complete ranking of efcient units. The DEA models that relax the condition for unit efciency are called super-efciency models. Super-efciency models are always based on removing the evaluated most efcient unit from the set of units. This removal leads to the modication of the efcient frontier, and the super-efciency is measured as a distance between the evaluated unit and a unit on the new efcient frontier. Of course several distance measures can be used this leads to different super-efciency denitions. The rst super-efciency DEA model was formulated in [1]. Its input oriented formulation (3) below is very close to the standard input oriented formulation of model (1). In this model, the weight q of the evaluated unit DMUq is equated to zero. This cannot inuence the efciency score of the inefcient units but the efciency score of the efcient units is not limited to 100% in this case. The input oriented formulation of the Andersen and Petersen model with a constant return to scale is as follows: minimize subject to
j =1,=q n

z = ,
n

xi j j + si = xiq , yi j j si+ = yiq ,

i = 1, 2, . . . , m , (3) i = 1, 2, . . . , r,

, s , s 0. The output oriented super-efciency model with constant or variable returns to scale can be formulated in a way similar to previous models. More information about DEA models can be found in Cooper [2]. 3. The interval AHP model The Analytic Hierarchy Process (AHP) is a powerful tool for the analysis of complex decision problems. The AHP organizes the decision problem as a hierarchical structure containing several levels. The rst (topmost) level denes the main goal of the decision problem and the last (lowest) level usually describes the decision alternatives or scenarios. The levels in between can contain secondary goals, criteria and subcriteria of the decision problem. The number of the levels is not limited, but in the typical case it does not exceed four or ve. Let us consider a simple three-level hierarchy that represents a standard decision problem with a nite set of alternatives n alternatives X 1 , X 2 , . . . , X n , are evaluated by k -criteria Y1 , Y2 , . . . , Yk , (Fig. 1). The decision maker expresses his/her preferences by comparing the importance of the elements on the given level with respect to an element of the preceding level. The decision makers judgements with respect to a given element are synthesized into the local priorities. They can express, e.g. the relative importance of criteria (weight coefcients in Fig. 1 denoted by v j , i = 1, 2, . . . , k ) or preference indices of the units with respect to the given criterion (wi j , i = 1, 2 . . . , n , j = 1, 2, . . . , k ). In a standard AHP model, the decision makers judgments on elements in a given level with respect to an element in the level above are organized into paired comparison matrices. The judgments are estimates of the preference between two elements of the lower level with respect to the element in the level above. Let us denote the paired comparison matrix A = {ai j | a ji = 1/ai j , ai j > 0, i , j = 1, 2, . . . , k }, where k is the number of elements in the particular comparison set of the lower level. Saaty [4,5] proposes to use ai j integers in the range 1 through 9 to express preference, where 1 means that the i -th and the j -th element are equally important and 9 means that the i -th element is absolutely more

j =1,=q +

1094

J. Jablonsky / Mathematical and Computer Modelling 46 (2007) 10911098

Fig. 1. Three-level hierarchy.

important than the j -th element. The local priorities are derived by solving the following eigenvector problem A.v = max v,
k

vi = 1,
i =1

(4)

where max is the largest eigenvalue of A and v is the normalized right eigenvector belonging to max . In the standard deterministic AHP approach, the decision maker always species point estimates that express his preference relations between two elements in the given hierarchical levels with respect to the parent element in the level above. It can often be very difcult for decision makers to do this. They may feel much better and closer to reality to have the possibility of expressing their preferences as interval estimates. For instance, instead of saying that the i -th element is four times as preferable as the j -th element, (s)he can assert that the i -th element is at least two but no more than ve times as preferable as the j -th element. The AHP model with interval decision maker judgements is usually called Interval AHP (IAHP) model. It is characterized by interval pairwise comparison matrices given as follows: 1 p12 , q12 p1k , q1k p21 , q21 1 p2k , q2k (5) A= . . . . . . . . . pk 1 , q k 1 pk 2 , q k 2 1 where pi j is the lower bound and qi j the upper bound for the preference relation (ai j ) between the i -th and j -th element. Due to the reciprocal nature of the pairwise comparison matrices, the relation pi j .q ji = 1 holds for all i , j = 1, 2, . . . , k . The judgments in the IAHP can be considered as random variables dened over the given interval. In this way, the IAHP changes from a deterministic model to a model with some stochastic features. That is why it cannot be analyzed in the traditional way by solving the eigenvector problem (4). It is necessary to look for new approaches that can take into account the stochastic features. The random variables for describing interval judgements can be selected from all available probabilistic distributions. We will use the uniform distribution dened over the interval pi j , qi j in our numerical experiments below. The preferences for the elements derived from matrix A are random variables. Their characteristics can be computed by several approaches we used a Monte Carlo simulation that is very simple, and offers lower and upper bounds for the preferences in a very short time. 4. DEA analysis versus IAHP model The DEA models are based on maximizing the individual efciency of the evaluated unit under the constraint that the efciency of all the other units is lower than 1 (100%). In this formulation, the variables of the model are the weights of the inputs and outputs, i.e. weights of the criteria in the sense of multiple criteria decision making. The DEA

J. Jablonsky / Mathematical and Computer Modelling 46 (2007) 10911098 Table 1 Pension funds criterion matrix # of cust. Allianz Credit Suisse CSOB Progres CSOB Stabilita Generali ING PF CP PF I CP PF II CS PF KB PF PF Ostrava PF Zemsky 106 611 18 304 23 346 225 518 401 285 19 14 Assets 4 095 22 592 452 8 508 789 9 767 6 348 12 441 10 954 11 776 935 468 Equity 77.0 549.0 56.0 298.6 74.0 289.1 290.7 522.5 223.5 441.6 71.0 87.9 Tot. costs 49.5 454.1 15.1 203.3 15.5 221.7 184.7 297.3 238.8 166.0 18.2 23.2 Appr. 1 3.00 3.36 4.30 2.30 3.00 4.00 3.34 3.10 2.64 3.40 2.44 4.01 Appr. 3 3.69 3.67 4.15 2.83 3.90 4.27 3.65 3.37 3.31 4.14 2.68 4.24

1095

Prot 1.29 5.22 1.13 10.87 0.45 0.26 6.83 6.90 1.10 6.40 0.04 2.03

model nds the best weights in order to maximize the efciency of the evaluated unit DMUq . In standard DEA models, the efcient units receive an efcient score of 1 (100%). With regard to the number of units of the evaluated set on one hand, and the number of inputs and outputs on the other hand, the number of efcient units can be relatively high. That is why several super-efciency models were proposed, in order to make it possible to classify the efcient units. In the super-efciency models, the units indicated originally as efcient take on distinct values of the super-efciency measure greater than one. Than, this measure can be used for their discrimination and classication. Our aim was to compare the results given by super-efciency DEA models with multiple criteria decision making methodology that can be represented very well by an AHP model. We used an IAHP model because it makes it possible to incorporate an appropriate level of uncertainty in the analysis that is typical for economic decision making problems. Both approaches will be demonstrated on a small numerical example with a real economic background. It is the problem of evaluating the efciency of the available pension funds in the Czech Republic. We worked with the data sets for 12 pension funds, each of them characterized by the following seven criteria (the data are from the year 2003): INP 1 the number of customers (thousands), INP 2 total assets (mil. CZK), INP 3 equity capital (mil. CZK), INP 4 total costs (mil. CZK), OUT 1 appreciation of the customer deposits for the last year (2003) (%), OUT 2 average appreciation of the customer deposits for the last three years (20012003) (%), OUT 3 net prot (mil. CZK).

For the DEA analysis, the rst four criteria were taken as inputs and the remaining ones as outputs of the model. The criterion matrix is given in Table 1. As it is possible to see from their assets, the funds listed in the previous table are different in nature. Four of them are very small (CSOB Progres, Generali, PF Ostrava and PF Zemsky), and the remaining ones are signicantly bigger. That is why we decided to analyze them separately. In Table 2, we present the results of our DEA analysis of both groups of funds (large and small). We used the basic envelopment DEA model with variable returns to scale with output orientation and the super-efciency model under the same assumptions. The rst column of Table 2 contains the efciency scores of the evaluated units the score is presented as a reciprocal value of the optimal score given by the model, because in output oriented models the score of inefcient units is greater than one. In other words, the higher score corresponds to a more efcient unit. The same holds for the super-efciency scores presented in the last column of Table 2. Of course, the super-efciency scores are available only for units indicated as efcient by the standard model. The word infeasible for the super-efciency score of the Allianz fund means that the corresponding VRS super-efciency model has no feasible solution. This situation can occur in VRS super-efcient models very often, and in this situation it is not possible to classify the efcient units. The IAHP model for the evaluation of efciency is very simple, and it is presented in Fig. 2. In the model, q1 is the total weight of inputs and q2 is the total weight of outputs, q1 + q2 = 1, the q i j , j = 1, 2, . . . , m , are the weights of single inputs, and q o j , j = 1, 2, . . . r , are the weights of single outputs. Preference indices

1096

J. Jablonsky / Mathematical and Computer Modelling 46 (2007) 10911098

Table 2 Efciency measures given by DEA models DEA/VRS model Large funds Allianz Credit Suisse CSOB Stabilita ING PF CP PF I CP PF II CS PF KB PF Small funds CSOB Progres Generali PF Ostrava PF Zemsky 100.00 93.72 64.06 100.00 Infeasible 100.00 96.12 100.00 100.00 100.00 95.64 82.14 100.00 Infeasible 159.24 119.87 122.40 Efciency score Super-eff score

112.40

Infeasible

Fig. 2. AHP model for evaluation of efciency.

u i j , i = 1, 2, . . . , n , j = 1, 2, . . . , m + r , express the preference of the i -th alternative (pension fund) with respect to the j -th input/output. The global preference indices of alternatives pi , i = 1, 2, . . . , n , are synthesized from previous hierarchical levels as follows:
m +r

pi =
j =1

ui j ,

i = 1, 2 , . . . , n .

In our analysis, the weights of the inputs and outputs were not derived by the AHP model, but they were either to set up directly as constants or optimized as variables of the proposed simulation model. In the model, the pairwise comparisons of alternatives (pension funds) with respect to all inputs and outputs were given as random variables with the uniform distribution dened on the interval a , b . The comparisons reect given criterion values, but in this way it is possible to use different returns to scale for different inputs and outputs. It is one of the advantages of this approach. Our numerical experiments were applied to a set of ve big pension funds identied as efcient by a DEA model with variable returns to scale. The pairwise comparisons of all alternatives with respect to the rst input (number of

J. Jablonsky / Mathematical and Computer Modelling 46 (2007) 10911098 Table 3 Pairwise comparisons of the IAHP model Number of customers Allianz CSOB Allianz CSOB ING CP PF KB PF 1 6, 8 1 1, 3 1, 2 ING 7, 9 1, 2 1 3, 5 2, 4 CP PF 3, 5 KB PF 5, 7 2, 3 1 Appreciation of customer deposits Allianz CSOB ING 1 4, 6 1, 3 1, 3 3, 5 1 5, 7 3, 5 3, 5 CP PF

1097

KB PF

3, 5 1

3, 5 1 1

Table 4 Comparison of super-efciency measures Pension funds Allianz CSOB Stabilita ING PF CP PF KB PF DEA/VRS super-eff. Infeasible 159.24 119.87 122.40 112.40 AHP I 0.4574 0.3608 0.3652 0.2123 0.2397 AHP II 0.3262 0.2583 0.2764 0.2056 0.2138 AHP III 0.3334 0.1449 0.1894 0.1699 0.1625

employees) and the rst output (appreciation of customer deposits) are given in Table 3 (free cells are the reciprocal values). The comparison matrices for the remaining inputs and outputs are given in the same way. The DEA models maximize the individual efciencies of evaluated units by looking for optimum weights of inputs and outputs. That is why we did not work with weights derived by the IAHP model, but rather tried to optimize the weights in order to reach the best efciency score of the evaluated alternative. We used the following requirements for the set of weights: 1. The sum of weights equals 1, all the weights have to be greater than 0.05 (AHP I). 2. The sum of weights of inputs equals 0.5, the same holds for the weights of outputs. All the weights have to be greater than 0.05 (AHP II). 3. All the weights are xed to value 1/(m + r ), i.e. 1/7 in our example (AHP III). The optimization run was carried out using Crystal Ball, which is an MS Excel add-in application for Monte Carlo experiments. Crystal Ball contains a special tool for optimization under stochastic conditions called OptQuest. This tool can nd the optimum values of variables (weights of inputs and outputs in our case) in a stochastic environment and can be modeled within MS Excel. The optimization criterion is the efciency score pi , i = 1, 2, . . . , n , of the evaluated alternative that is to be maximized. Because the efciency score under our stochastic conditions is a random variable, we tried to maximize its mean value. We always used a ve-minute optimization run for all the alternatives, with 100 trials per simulation. The results are presented in Table 4. The rst column of Table 4 contains the superefciency scores computed by the Andersen and Petersen DEA model with variable returns to scale; the remaining three columns contain the maximized efciency score of pension funds given by the IAHP model with different weight constraints (weight sets I, II and III). All DEA results were computed by means of the original MS Excel add-in application that can be downloaded from the authors web page some information about it can be found in Jablonsky [3]. Very nice and powerful DEA solver is included as the part of the book in Zhu [6]. The DEA/VRS model is not able to evaluate the Allianz fund; the best among the others is the CSOB Stabilita fund and the worst is the KB pension fund. The results given by the IAHP models are quite different. If we consider the same weights for all the inputs and outputs (AHP III), the Allianz fund is classied at the top and the remaining funds are very close each other. For the remaining set of weights (AHP I and AHP II), the funds are grouped in almost identical ways. The best is the Allianz fund, the mean efciency score of the CSOB and ING pension funds is more or less the same, and at the bottom of the ranking are CP PF and KB PF the last two funds. The optimal weights computed by the OptQuest application for the set of weight constraints are presented in Table 5. 5. Conclusions The aim of the paper was to determine how the AHP models can be used for the efcient evaluation of production units, and to compare the results given by the proposed Interval AHP model with the efciency scores computed by

1098 Table 5 Optimal weights given by OptQuest

J. Jablonsky / Mathematical and Computer Modelling 46 (2007) 10911098

Allianz # of customers Total assets Equity Total costs Appreciation 1 Appreciation 3 Prot 0.2574 0.1116 0.3266 0.1545 0.0500 0.0500 0.0500

CSOB 0.0509 0.0500 0.0500 0.0500 0.0500 0.0510 0.6981

ING PF 0.0500 0.0500 0.0500 0.0500 0.4592 0.2908 0.0500

CP PF 0.0500 0.7000 0.0500 0.0500 0.0500 0.0500 0.0500

KB PF 0.0500 0.0500 0.0500 0.0500 0.0500 0.5676 0.1824

DEA models. In contrast to super-efciency DEA models, the advantage of the IAHP approach consists in several points: the IAHP model can use different scales for different inputs and outputs according to the decision makers preferences, in super-efciency DEA models with VRS, not all the units receive their super-efciency scores (as the solutions are not always feasible), the IAHP model can deal with categorial inputs and outputs without any transformation, the IAHP model offers the possibility of performing a sensitivity analysis on the results with respect to the inputs and/or outputs. The main disadvantage of the AHP (IAHP) models compared to the DEA models consists in preparing the data for the analysis (pairwise comparison matrices) and in time-consuming length of the optimization analysis. Nevertheless, by using the AHP model, the decision maker can receive new information useful for the global analysis of the efciency of the evaluated set of units. Future research in this eld will be focused on comparing other super-efciency DEA models (slack based models) with IAHP models and other multiple criteria decision making techniques. Several real-life economic applications will serve as background for numerical experiments. Acknowledgement The research is partially supported by the Grant Agency of the Czech Republic project no. 402/06/0150. References
[1] [2] [3] [4] [5] [6] P. Andersen, N.C. Petersen, A procedure for ranking efcient units in data envelopment analysis, Management Science 39 (1993) 12611264. W.W. Cooper, L.M. Seiford, K. Tone, Data Envelopment Analysis, Kluwer Publ., Boston, 2000. J. Jablonsky, Models for evaluation of efciency of production units, Politicka Ekonomie 2 (2004) 206220 (in Czech). T.L. Saaty, The Analytic Hierarchy Process, RWS Publications, Pittsburgh, 1990. T.L. Saaty, Decision Making with Dependence and Feedback: The Analytic Network Process, RWS Publications, Pittsburgh, 1996. J. Zhu, Quantitative Models for Performance Evaluation and Benchmarking, Kluwer Publ., Boston, 2003.