PERFORMANCE MANAGEMENT RESEARCH UNIT (PMRU) An international survey of performance measurement in airports Graham Francis*, Jackie Fry* and

Ian Humphreys** September 2001

01/4

ISBN 0 7492 45433

The Open University Graham Francis, Jackie Fry, Ian Humphreys Loughborough University
*

Performance Management Research Unit, Open University Business School, Walton Hall, Milton Keynes, MK7 6AA g.a.j.francis@open.ac.uk j.fry@open.ac.uk **Transport Studies Group, Department of Civil and Building Engineering, Loughborough University, Loughborough, Leicestershire, LE11 3TU i.m.humphreys@lboro.ac.uk

Abstract
The seemingly relentless growth in air traffic and the introduction of commercial and private models of airport ownership has changed the scope and significance of performance measurement for airport management, regulators and stakeholders. The object of this paper is to report the findings of a questionnaire survey which examined the nature and prevalence of the performance measurement of airports. Previous studies have examined specific airports or groups of airports and compared performance. This research assesses the way in which performance measures are used by management at the worlds busiest passenger airports using empirical evidence collected principally through the means of a world-wide questionnaire survey of the 200 largest passenger airports. A rich picture of different performance measurement practises was discovered.

1. Introduction
Performance measurement is a critical activity at both the individual airport and the wider system levels. Managers and Government need to have information to measure efficiency from a financial and an operational perspective (Doganis, 1992), to evaluate alternative investment strategies, to monitor airport activity from a safety perspective and to monitor environmental impact. Performance measurement can be used by management to identify areas that are performing well as well as those that are not. Once performance is known, management can examine the underlying processes taking place so that appropriate corrective action can be proposed.

The study of performance measurement in airports is becoming increasingly important because of issues surrounding the social, economic and environmental sustainability of airport expansion in the face of forecast growth in traffic. Global air passenger traffic is forecast to increase by an average of 4.6 to 4.9 percent per annum between 1999 and 2020 (Boeing, 2001; Airbus, 2000; ICAO, 2000). The forecasts unrealistically assume an unconstrained availability of infrastructure to accommodate growth. However, despite the scale of development, congestion, particularly at major airports in Europe and the US, is likely to continue. There is growing political, social and environmental opposition to expansion. The nature of the need for measurement is also changing with

the introduction of new operational contexts with many airports seeing a change in their raison dtre from public service to the maximisation of shareholder value.

This research builds on earlier work by the authors including a series of interviews of airport managers (see Humphreys and Francis, 2000a, 2000b). This paper starts with a review of the literature on the performance measurement of airports and then describes the methods used to collect and analyse data. The paper discusses the nature, prevalence and consequences of current performance measurement systems. The conclusions consider the implications of the performance measurement practices and point to areas of potential future research.

2. The development of performance measurement of airports

In the air transport industry the use of performance indicators has been led by airline management with well established industry measures such as cost per available seat kilometre, yield per revenue passenger kilometre and labour productivity per available tonne kilometre (Doganis, 1992; Caves, Christensen, and Treathaway, 1983). Measures that developed were often based around the Work Load Unit (WLU), defined as one passenger processed or 100kg of freight handled (Doganis, 1978). This measure was adopted by the airlines around 20 years ago to provide a single measure of output for passenger and freight business (for example see CIPFA, 1980 and BIE, 1994). Typical measures used included: total cost per WLU; operating cost per WLU; depreciation cost per WLU; labour cost per WLU; WLU per employee; WLU per unit asset value; total revenue per WLU and aeronautical revenue per WLU. (Doganis, 1978, 1983, 1992; Doganis and Graham, 1987; Graham 1991, 1999; BIE, 1994). In the UK, annual airport performance results were collated on a national basis and made available to the public by the Chartered Institute of Public Finance and Accountancy. Table 2.1 lists the measures reported by Graham (1999) and Doganis (1992). The nature and prevalence of these measures form part of this research; the FLAP group measures were from the authors earlier research.

Table 2.1: European airport performance categories and metrics Measures used by airports Graham and Doganis measures Measures of overall cost performance Total cost per WLU Operating cost per WLU Depreciation cost per WLU Labour cost per WLU Depreciation share of total costs Labour share of total costs Labour productivity measures WLU per employee Total revenue per employee Value added per employee Value added per unit labour cost Capital productivity measures WLU per unit asset value Revenue per unit asset value Asset value per employee Revenue generation performance Total revenue per WLU Aeronautical revenue per WLU Non-aeronautical revenue per WLU FLAP group1 Revenue Traffic income per passenger Traffic income per WLU Traffic income per turnover % Commercial income per passenger Concession income per passenger Duty and Tax free inc per Int Dep Passenger Other concession income per passenger Property income per passenger Property income per WLU Cost Staff cost/employee Passenger /employee num WLU per employee num Staff cost per passenger Staff cost per WLU Other direct costs per passenger Other direct costs per WLU Contribution Contribution per passenger Contribution per WLU Contribution per employee Unadjusted reported accounts Profits/equity % Debt/equity% Cash flow/cap-x
Sources: Authors and adapted from Doganis (1992) and Graham (1999).

The FLAP group are made up of representatives of Frankfurt, London, Amsterdam, and Paris Airports

The operational performance of airport infrastructure was measured against design and level of service criteria standards defined by the International Civil Aviation Organisation and applied by specific Government aviation agencies. Many authors have specified the different standards (for example Ashford, 1988; Muller and Gosling, 1991) and some have pointed out that certain standards appear to be very generous in terms of space per passenger (Kazda and Caves, 2000). This measurement technique was borrowed and adapted from the field of highway design. Airport terminal design standards were expressed in terms of the space offered to a user at each part of the airport facility, i.e. square metres at each facility per passenger per hour (Ashford, Stanton and Moore, 1995). The level of service alters depending upon the degree of crowding at each facility. This gave an aggregate view of passenger service levels within the airport terminal. Current developments in airport performance measurement are now considered.

Airports have traditionally been compared to their peers (Graham, 2001) but have now started to recognise the potential for benchmarking against other airports to improve their competitive position through the identification and adoption of best practices (Centre for Airport Studies, 1998). A recent development has been the performance measurement and benchmarking of retail activities on a global scale (Cerovic, 1998; Favotto, 2001). The increased importance of benchmarking comparisons for airports is reflected by the International Air Transport Associations (IATA) latest service that allows any airport in the world to compare their own service quality performance against the performance of the 48 airports that participated in its Global Airport Monitor service quality survey. The annual IATA survey measures airport performance against 19 different service criteria by interviewing travellers at a range of airports (Hegendorfer and Morris, 2000; IATA, 2001).

Airports have begun to measure and compare quality of service measures with each other. The world airports body, Airports Council International (ACI) has begun to facilitate comparisons and has produced a survey of service standards and measurements used at 120 of its member airports (ACI, 2000a).

The latest developments in performance measurement respond to the increasing importance of social and environmental opposition to air transport. IATA have begun work on a benchmarking study of environmental measures that is reported to include measures related to airports (Dobbie, 2001) and sustainability indicators related to the air transport system in the UK are currently being developed under the UK Government Sustainable Cities research programme (Upham and Yang, 2001).

Performance measurement data has been and is being collected and published in a variety of forms on a national and a global basis. So far surveys have concentrated on what measures are being used. However there appears to be no literature that surveys how these measures are used internally by airport managers. This survey was undertaken in order to address this gap in the literature in terms of the use of performance measures by airport managers and to assess the extent to which different measures were in practical use at the 200 busiest airports in the world.

3. Research methods
3.1. Pilot survey The questionnaire was pretested on six airport management short course delegates at Loughborough University in May 2000. The sample for the pilot survey was taken from the Loughborough University Short Course Database. A questionnaire and a covering letter were sent to previous attendees from UK airports on 15th May 2000. Repeat mailings are known to increase the response rate (Dillman, 2000, p. 149) and therefore each questionnaire was given a unique identification number to ensure repeat mailings were only sent to nonrespondents. Two repeat mailings were sent to nonrespondents on 5th and 17th June 2000. Forty-five questionnaires were sent out, two were returned as undeliverable, four were returned blank and 15 were returned completed, a response rate of 44 percent.

Twelve different airports were represented, six privately owned, four publicly owned and two part private, part publicly owned. One handled between ten and twenty million passengers per annum, three between five and ten million passengers, four between one and five million and four less than one million.

3.2. World survey The set of airports sampled was the top 200 airports as ranked by the ACI in terms of total passengers for 1999. The information was accessed on the ACI Web site (ACI, 2000b). The top 200 were chosen as these airports have approximately three million passengers or more per annum. Doganis (1992) and Graham (2001) have indicated that up to this threshold there are significant economies of scale for airports in terms of the unit cost per passenger or WLU unit handled.

Each questionnaire sent out was given a unique identification number to ensure repeat mailings were only sent to nonrespondents. A copy of the questionnaire and a covering letter were sent out on the 15th August 2000, 26th September 2000 and 7th November 2000. Copies of the questionnaire and covering letters are in Appendix 1.

Two hundred questionnaires were sent out in the first mailing. Some questionnaires were returned as undeliverable. Attempts were made to ensure the address was correct and the questionnaires resent. Despite these efforts, five remained undeliverable. Of the remaining 195 questionnaires, four airports declined to participate and 58 were returned completed, a response rate of 32 percent.

3.2.1

Demographics and nonresponse bias

The respondent airports and the airports in the sample were classified into geographic regions using the categories defined by ACI: Europe, North America, Latin America and the Caribbean, Asia, Africa and the Pacific (Table 3.1) The profile of the respondents was then compared to the profile of the overall sample. In order to perform a Chi-square test, the categories of Latin America, Asia, Africa and Pacific were combined into a rest of world category so that the expected values of the categories were greater than 5. The Chi-square test revealed that the profile of the respondents was not significantly different to the profile of the sample at the 5 percent level (2=4.90, ns). However it should be noted that the actual probability was 0.09. It can be seen from Table 3.1 that North America is proportionally over represented and the rest of the world under represented. This high response rate by airports in North America may be a

consequence of the strong public utility ethos at North American airports with respect to public information provision.

Table 3.1: Geographic profiles of the respondents and the sample Percentage of sample airports North America Europe Pacific Asia Latin America/Caribbean Africa 34 40 11 6 6 3 Percentage of respondents 48 38 5 5 2 2

The profile of the total number of passengers handled last year in the sample and the respondents is shown in Table 3.2. The profile of the respondents was then compared to the profile of the overall sample. The Chi-square test revealed that the profile of the respondents was not significantly different to the profile of the sample at the 5 percent level (2=0.77, ns). The profile of the respondents is a very good match to the profile of the sample.

Table 3.2: Number of passengers handled by the airports responding to the survey Passengers handled/ million 1 to 4* 5 to 9 10 to 19 20 and above
*

Percentage of sample airports 30 29 19 22

Percentage of respondents 29 24 23 24

Only includes up to the 200th busiest airport

The pattern of airport ownership amongst the respondents is shown in Table 3.3. The Chi-square test revealed that the profile of the respondents is significantly different to the profile of the sample at the 5 percent level (2=9.38, p<0.05). It can be seen from

Table 3.3 that airports in public ownership are proportionally under represented and airports in private ownership are over represented.

Table 3.3: Ownership structures of airports responding to survey Type of ownership Percentage of sample airports Public Private Part private part public 80 8 12 Percentage of respondents 67 19 14

Table 3.4 shows the break down of job function of the respondents to this survey.

Table 3.4: Job function of the respondents Job function Percentage of respondents Operations Administration Finance CEO/Director/senior management Planning Concessions management 30 30 14 10 9 7

In conclusion the profile of the respondents is a reasonable match to the original sample. However when interpreting the responses to the questionnaire it should be borne in mind that the views of North American airports and airports in private ownership are over represented.

4. The nature and prevalence of performance measurement at airports

Airports are very complex dynamic organisations consisting of many interacting parts including: passengers, airlines, handling agents, ground transportation service providers,

other aviation related activity and the interests of the regional and national economy. The range of different performance measures reflects this complex context. There has been a tendency to search for measures to simplify airport performance so that league tables can be drawn up and comparisons made. However in simplifying the indicator, the meaning and therefore the usefulness to management or government disappears. There is a tendency to use measures that are easy to monitor as opposed to what is important to measure (see Gosling, 1999, 2000 and Humphreys and Francis, 2000c).

Table 4.1 shows the range of performance measures used by the respondents. Best Practice Benchmarking was the most popular technique used by airports with almost half (46 percent) engaged in this activity. Various quality improvement techniques such as Total Quality Management (41 percent) and Quality Management Systems (23 percent) were employed at airports. A specific financial tool, Activity Based Costing was used by 36 percent of respondents and a broader measurement system, the Balanced Scorecard was utilised by one quarter of the airports, as were measures undertaken as part of Environmental Management Systems (27 percent).

Table 4.1: Performance management techniques used by respondents Technique Percentage use by respondents* Best Practice Benchmarking Total Quality Management (TQM) Activity Based Costing Environmental Management Systems (eg ISO14000) Balanced Scorecard Business Process Reengineering Quality Management Systems (eg ISO9000/BS5750or similar) Business Excellence Model / EFQM Value Based Management Malcolm Baldridge Award
*

46 41 36 27 25 23 23 12 9 5

Note that respondents could use more than one method

10

A variety of methods were used by each airport to collect performance measurement data. 95 percent of those who responded used surveys, 25 percent interviews and 20 percent used consultants. It might be expected that privately owned airports, which are over represented amongst respondents, would be more financially driven. The following sections explore performance measurement in the broad categories of financial, quality of service and environmental.

4.1. Financial Financial performance measures used at respondent airports in the main reflected the measures reported in the general literature (Table 4.2). A good level of awareness regarding whether or not certain measures were used was evident with 5 percent or less of respondents being unaware of the use of measures for all but two performance metrics. Perhaps this reflects the significance placed on financial measures in todays context of airports being run to commercial objectives (Humphreys, Francis and Fry, 2001). The focus may have been influenced by the proportion of privately owned airports amongst the respondents, but this does not alter the underlying trend.

The significance of commercial revenue for airports identified in the literature was supported by survey results. Non aeronautical income per passenger was deemed to be the most useful of the financial measures, in line with the trend towards airport management adopting a more commercial focus. In keeping with the literature (see Table 2.1, Graham, 1999 and Humphreys and Francis, 2000b), the most popular measures were those that were easiest to measure, income per passenger and expenditure per passenger. Measures that are more useful but harder to calculate, such as value added per employee were used by only 23 percent of respondents and scored well in terms of perceived usefulness (Table 4.2).

The relatively low use of total revenue per WLU (Table 4.2), and its rating as having a low usefulness to managers relative to other financial measures could reflect the lack of relevance to management of a financial measure that combines two outputs (passengers and freight) that have very different inputs (Humphreys and Francis, 2000a). WLUs are

11

perhaps more relevant as performance measures to airlines rather than airports who gain relatively little revenue from belly-hold freight activities compared to passengers.

Table 4.2: Financial performance measures used by respondents Performance measure Used Not used % Income per passenger Expenditure per passenger Concession revenue per m2 Non aeronautical income per passenger Aeronautical income per passenger Revenue to expenditure ratio Income per employee Capital expenditure per passenger Total revenue per Work Load Unit Value added per employee Other
*

Dont know % 2 2 0 4 4 6 2 7 5 5 0

Usefulness of measure* Mean 4.2 3.9 4.1 4.3 4.0 3.9 3.7 3.2 3.6 4.0 4.8 S 0.8 1.1 1.0 0.8 1.2 1.1 1.1 1.5 1.0 1.0 0.5

% 13 13 22 19 20 21 37 61 63 72 27

85 85 78 77 76 73 61 32 32 23 73

Scale: 1=Not useful to 5=Very useful, S=Standard Deviation

Table 4.3 shows the relative reported use of a variety of financial performance measures. Whilst more traditional measures such as budgets, profitability and cash flow were the most commonly used, other measures were also in use at some airports. The questionnaire also asked about which measures were being considered and relatively new measures such as Balanced Scorecard, SVA, EVA were reported as being considered by as many if not more airports as already using them, indicating likely future growth in the popularity of these measures. SVA and EVA in particular may be increasingly popular with the continuing commercialisation of airports.

12

Table 4.3: Use of financial performance measures Used Being considered % Ability to stay within budget Balanced Scorecard Economic Value Added (EVA) Residual Income Return on Capital Employed Shareholder Value Analysis (SVA) Target cash flow Target profits Value drivers 98 23 12.5 17 51 10 84 82 15 % 0 23 12.5 14 9.5 19 10 2 18 Not being considered % 0 38 52.5 49 30 57 2 14 46 Not aware of % 2 16 22.5 20 9.5 14 4 2 21

Table 4.4 triangulates the findings reported in Table 4.3 with a survey by Francis and Minchington (2000) and the authors UK pilot study. The airports in this survey showed a slightly different pattern to the Francis and Minchington (2000) survey of large UK private sector companies. This was perhaps to be expected as the survey was across a range of sectors. Table 4.5 shows how the prevalence of financial measures varied across these sectors. Whilst ability to stay in budget was used by virtually all organisations reported in the Francis and Minchington (2000) survey, slightly fewer airports reported using target profits. This undoubtedly reflects the varied ownership status of the sample with many airports not being required or expected to be profitable.

4.2. Quality of service measures The general perceived trend towards airports focussing on service quality and monitoring customer satisfaction reported by Hegendorfer and Morris (2000) was supported by the findings that quality improvement processes, of one kind or another such as TQM (41 percent), Quality Management Systems (23 percent) and Business Excellence (12 percent) were frequently being applied. Moreover this is evidence that many airports are actively striving to improve quality rather than merely measure it.

13

Table 4.4: Comparison of financial performance measures currently in use World airports % Ability to stay within budget Balanced Scorecard Economic Value Added (EVA) Residual Income Return on Capital Employed Shareholder Value Analysis (SVA) Target cash flow Target profits Value drivers 98 23 12.5 17 51 10 84 82 15 UK pilot % 88 38 0 0 66 29 100 100 0 Minchington and Francis (2000) (all sectors) % 99 24 10 6 71 15 70 94 28

Previously the extent of usage of such models has not been widely reported, but usage reflects the focus on customer satisfaction and other quality issues which may reflect the increasingly commercial agenda of the worlds airports (Ashford, 1999).

Respondents were asked to specify which quality of service indicators they personally used. 47 percent of respondents used a form of Quality of Service Monitor, 9 percent used passenger comment feedback cards, 34 percent used airport measured operational service performance and 12.5 percent did not personally use any.

The trend towards focussing on service quality is evident once again. The majority of managers at airports are formally measuring service quality by a systematic service monitor (47 percent) or some other form (34 percent).

14

Table 4.5: Comparison of financial performance measures currently in use by sector (Minchington and Francis, 2000) survey World airports % Ability to stay within budget Balanced Scorecard Economic Value Added (EVA) Residual Income Return on Capital Employed Shareholder Value Analysis (SVA) Target cash flow Target profits Value drivers 98 23 12.5 17 51 10 84 82 15 UK pilot % 88 38 0 0 66 29 100 100 0 Manufacturing % 99 21 12 5 80 13 72 92 20 Financial % 100 42 6 6 53 25 25 94 41 Utilities % 100 36 0 0 70 0 82 100 50 Retail % 100 26 17 10 65 18 78 96 39 Service % 100 18 7 3 53 12 69 96 38

Table 4.6: Quality of service performance measures used by respondents Performance measure Used Not used % Baggage delivery time Quality of signage / ease of finding way Check in waiting time Walking distances Availability of trolleys Other
*

Dont know % 0 2 0 2 6 0

Usefulness of measure* Mean 4.5 4.5 4.4 4.1 4.3 4.8 S 0.6 0.8 0.7 1.1 0.8 0.4

% 22 24 31 36 39 17

78 76 69 62 55 83

Scale: 1=Not useful to 5=Very useful, S=Standard Deviation

Other measures used are deemed to be useful to the airports that use them. An interesting point is the use of check in waiting time as a measure by airports because this typically depends on the airline and how many staff they allocate to this task and how many check-in desks they rent from the airport. It is possible that this is measured by airport management to monitor airline behaviour and the implications of this for the level of service delivered to passengers in the terminal and the impacts on passenger flow and space requirements in different parts of the terminal. Given expansion pressures, airports managers may need to monitor airline behaviour in order to manage the terminal efficiently now and to assess future capacity requirements realistically.

4.3. Environmental measures There has been a trend towards airports collecting more performance measurement data related to environmental impact (Humphreys and Francis 2000d). Interestingly, the survey reveals that performance measures related to officially accredited Environmental Management Systems accounted for 27 percent (Table 4.1). This result is surprisingly high on the one hand, it was not expected that airports had gone down the Environmental Management Systems route to this extent. On the other hand, given the necessity of airports to show an independent accountability regarding environmental impact such a result can be logically accounted for. In the face of airport congestion and the need to expand in an environmentally responsible way to cater for a forecast

16

doubling in traffic over the next 12 years, organised environmental measurement systems are likely to increase in popularity for airports.

Table 4.7: Environmental indicators used by respondents Performance measure Used Not used % Number of community complaints about operations Percentage of homes subjected to a specified noise level Percentage of passenger trips to the airport by public transport Percentage of waste recycled per annum Other
*

Dont know % 3

Usefulness of measure* Mean 4.2 S 0.9

% 13

84

67

28

4.3

0.9

53

43

4.5

0.9

50

39

11

4.1

0.9

86

4.7

0.5

Scale: 1=Not useful to 5=Very useful, S=Standard Deviation

Environmental measures used by airports were seen on the whole to be useful. Number of community complaints was the most popular measure (84 percent, see Table 4.7). This is a very significant finding because previously the degree of use of this measure and its perceived usefulness at the worlds busiest (and therefore often noisiest) airports was previously unknown. Recently the validity of the relationship between complaints and airport operations has been called into question by community groups, airports and academics (Hulme et al., 2001). Perhaps this is an example of an easy to measure number being used by airports to chart the opposition to their operations and the difficulty of trying to measure opposition in more detail. Reasons for complaints, their validity and effectiveness of action taken to address community concerns may be difficult to measure but will surely become measures that are important to explore if airports wish to expand to accommodate the forecast growth in operations (ICAO, 2000).

17

The noise level experienced by surrounding homes was used by over two thirds of the respondents (67 percent). This result was expected because noise from aircraft operations is often related to the established and ongoing community annoyance (number of complaints). Other environmental measures such as the significance of public transport (53 percent) and the significance of recycling (50 percent) (Table 4.7) are set to become more important as airports seek to accommodate forecast air traffic growth in an increasingly environmentally responsible manner. There was an interesting divide between North American and European airports on the use of performance measures for public transport and recycling. The use of the measures, percentage of passenger trips to the airport by public transport and the percentage of waste recycled per annum, are much more prevalent at European airports. This may be a reflection of the differing operational and political contexts of the two continents.

4.4. Benchmarking Airports appear to reflect the trend found in a broader cross section of industry that larger organisations are more likely to carry out some form of benchmarking activity. Airports with more than 5 million passengers per annum were almost twice as likely to engage in some form of benchmarking activity, than those with less than 5 million passengers per annum (83 percent compared to 44 percent).

The focus on benchmarking was with similar airports, 72 percent of respondents who were engaged in benchmarking used comparative performance data and most (97 percent) compared themselves to other airports. Only two airports indicated that data was used from a different industry sector for a similar function, for example comparing data on trolleys with large shopping centres. One airport compared their performance data with their own previously collected data. Some 42 percent of respondents compared service qualitative data and 50 percent of respondents compared financial data.

The questionnaire invited respondents to locate their experience of benchmarking on a series of 7-point Likert scales. Benchmarking among the survey airports was almost exclusively undertaken between similar airports (Table 4.8), and airports were more

18

concerned with measurement than process improvement (Table 4.9) and with financial rather than non-financial measures (Table 4.10). Experience of internal and external comparisons and benchmarking experience related to specific tasks and general practices were evenly split (see Tables 4.11 and 4.12).

Table 4.8: Locating benchmarking experience: similar to dissimilar Similar 1

!

3
"

4 16%

5 3%
!

6 3% 6%

7 0%
"

Dissimilar

14% 40% 24% 78

Table 4.9: Locating benchmarking experience: process improvement to measurement Improvement 1 3%

!

2 5% 27%

3 19%
"

4 8%

5
!

7 8%
"

Measurement

19% 38% 65%

Table 4.10: Locating benchmarking experience: financial to non-financial Financial 1 8%

!

3
"

4 32%

5 3%
!

6 10% 18%

7 5%
"

Non-financial

21% 21% 50%

Table 4.11: Locating benchmarking experience: internal to external Internal 1 8%

!

2 24% 40%

3 8%
"

External

21% 10.5 %
!

18% 10.5 % " 39%

19

Table 4.12: Locating benchmarking experience: specific tasks to general practices Specific 1 5%
!

3
"

5
!

7 3%
"

General

16% 16% 37%

22% 24% 14% 41%

In a number of comments made by respondents three themes related to airport experience of benchmarking prevailed. Firstly, that benchmarking activity was Just implemented and a New experience for a number of the airports, possibly a

reflection on the new commercially focused approach to airport management introducing benchmarking practices.

Benchmarking was seen as difficult to do particularly with respect to finding comparable data and being able to do like with like comparisons. The view expressed by one manager captures the feeling of many survey respondents that for benchmarking it is Challenging to get needed information and to ensure apples to apples comparisons. Airport groups, particularly those that are building global portfolios have access to data problems reduced and could seize this opportunity to exploit inter airport learning across a variety of geographic contexts.

Finally, respondents claimed a wide variety of practical benefits as a result of benchmarking activity such as: We are able to reduce service time and dwell time, We use it to drive training programmes, ...to monitor and manage concession behaviour and for identifying models of best practice. However one airport manager reflected that in their experience benchmarking identifies areas for improvement (but does not help on how to improve them!).

Airport managers are making comparisons with similar airports to their own, a trend that may be consistent with the fact that 70 percent of respondents who benchmarked used it mainly for measurement purposes not process improvement. Given these figures, the respondents may be able to gain more benefits in terms of performance improvement from benchmarking in the future if they place more emphasis on learning from the processes that are generating the relative measures of performance. The

20

tendency is to focus on results more than the underlying processes (Holloway et al., 1999). Further benefits from benchmarking may be realised if managers consider looking for exemplar practices of the processes they are trying to manage and improve at dissimilar airports or even generic examples within other industries. Looking at the management issues from this wider perspective may be more risky but can give greater benefits.

4.5. Other measurement issues Performance measures were frequently used to improve performance of the airport by allowing it to monitor performance and allocate appropriate management attention. As airport managers commented performance measures Give us an indication of where to concentrate our efforts. Another airport manager stated performance measures Highlight areas of under performance and undertake investigations into reasons for variances.

External factors, for example regulation or European Union directives, influenced the choice of the new measures for 40 percent of respondents. 60 percent said internal factors influenced the choice of the new measures, for example passenger expectations. The split possibly reflects the pressures for increased customer focus and the growth of regulation at airports that has accompanied new commercialised and privatised ownership forms. For the vast majority of respondents the decision about which performance measures to use, whether internal or external, was taken internally (98 percent) rather than being imposed externally.

The questionnaire survey also included questions about the introduction of new performances measures, 58 percent of respondents said their organisations had introduced new performance measures in the last two years. The size of the airport seems to be an important factor here. Of those airports that handle between one and twenty million passengers per annum, 50 percent had introduced new performance measures. Of those who handle more than twenty million passengers per annum 85 percent had introduced new performance measures. Airport ownership structure seems to have less impact than size as to whether or not any new performance measures had

21

been adopted in the last two years: private (64 percent), public (57 percent) or partprivate part-public (57 percent). The Balanced Scorecard was the most commonly introduced new measure and probably reflects attempts to balance airports various financial and non-financial objectives. Management consultants also had some impact on the choice of methods as indicated in Table 4.13 below.

Table 4.13: Influence of management consultants on the introduction of new performance measures Percentage of respondents Very influential Some influence Not influential Management consultants not used 23.5 29 21 26.5

It may follow that larger airports might be more likely to introduce new measures as the rapid growth of traffic is more likely to affect them in a more profound way just because of their scale. In some senses there is more for managers to manage at larger airports and so it is important for them to be introducing new measures to monitor new trends in the context of growth, commercial focus and environmental responsibility. A focus for management was the tangible benefit most often cited (80 percent) by those who had introduced new performance measures within the last two years. Perhaps at larger airports the likelihood of new measures producing tangible benefits is likely to be greater.

There may also be a sense that the larger airports have the greater resources and therefore more capacity to introduce new measures. Airports with less than 20 million passengers per annum had far fewer new performance measures than might have been expected. The growth and other changes in the airport operating environment are likely to present a range of different but not fewer management problems for the smaller airports in the sample, all of which require monitoring. If the issue is one of resources

22

then perhaps management need to focus on the possible benefits of new performance measures.

Performance and its relationship to systems of pay was investigated. In summary 56 percent of respondents indicated their airport had performance related pay and in those who did applied it to all staff in 64 percent of cases and manger level or higher in 36 percent of cases. Privately owned airports show a greater tendency to have performance related pay (73 percent) than publicly owned (49 percent) or part-private part-publicly owned airports (67 percent). Those publicly owned airports who have performance related pay generally have it for all staff (73 percent) rather than for just manager level or higher. In contrast, 43 percent of privately owned airports who have performance related pay have it for all staff. These results highlight the effects of the growing commercial focus of airports, with private operators tending to place more emphasis on performance related pay and to differentiate between the manager level of employee, whereas publicly owned airports place less emphasis on performance related pay and when they do engage in this it is more likely to apply to all levels of staff. What difference this makes to airport performance is a subject that requires further investigation.

5. Conclusions
A similar range of dynamic challenges face airport management round the world which include airport congestion, rapid traffic growth, increased commercial business focus for airports, environmental pressures, increased regulation and the globalisation of airline and airport activity. What is more, these challenges are likely to grow in magnitude. The need for effective, dynamic and relevant performance measurement has never been greater in order for management to manage effectively. Evolution and improvement of performance measurement systems and their application can play a key role for airport managers, whatever their context. The survey offered some evidence that the increasing commercialisation of airports is impacting on the nature of performance measurement systems in use by airports.

23

The motivation for this paper was to gain a better understanding of performance measurement practices in airports around the world. The survey revealed a rich picture of different practices from which certain general trends could be discerned. This survey sought to quantify for the first time the extent to which a range of performance measures are being used across a range of airports in terms of ownership, size and geographical context. Some evidence was found that size, ownership and location can impact on choice of performance measurement systems. Significantly the survey reveals that most measures are perceived to be useful by their users, albeit to differing degrees, even if some measures produced information of limited value. In order to gain a step change in airport performance, airports may have to shed the view of merely comparing themselves within their own industry and look outside the industry for synergy and learning from processes in other industries.

A rich vein of survey data has been established that can be analysed in more detail with respect to particular performance measurement issues. In terms of further research, whilst this study has increased knowledge about the nature and prevalence of performance measurement practices by airports, further interviews and case studies will add to the understanding of how measures are used. The monitoring of measurement trends over time might reflect how management in different parts of the world tackle problems and may provide a basis to share knowledge, understanding and its application in different contexts.

6. Acknowledgements
The authors wish to thank all those who took part in this survey, Jacky Holloway, Open University Business School and Robert Caves, Transport Studies Group, Department of Civil and Building Engineering, Loughborough University. The Performance Management Research Unit, Open University Business School funded this project.

7. References
ACI (2000a) Quality of Service at Airports: Standards and measurements. ACI, Geneva.

24

ACI (2000b) World Airports Ranked by Passenger Throughput 1999. www.airports.org/traffic/td_passengers_doc.html. Accessed May 16, 2000. Airbus (2000) Global Market Forecast 2000-2019. Airbus, Blagnac Cedex. Ashford, N. (1988) Level of Service Design Concept for Airport Passenger Terminals a European View. Transport Planning and Technology, Vol. 12, pp. 5-21. Ashford, N. (1999) Airport Finance. Loughborough Consultancy, Loughborough. Ashford, N., Stanton, M. and Moore, C. (1995) Airport Operations. Wiley, New York. BIE (1994) International Performance Measurement Indicators: Aviation. Bureau of Industry Economics Research Report 59, Australian Government, Canberra. Boeing (2001) Current Market Outlook 2000, www.boeing.com. Accessed 15th April, 2001. Caves, D., Christensen, L., and Trethaway, M. (1983) Productivity Performance of US Trunk and Local Service Airlines in an Era of Deregulation. Economic Inquiry, Vol. 21, pp. 312-324. Centre for Airport Studies. (1988) Airport Retail Study. Paper presented to the ACI World Conference, Sydney. Cerovic, M. (1998) Global Airport Retailing. Reuters Business Insight, London. CIPFA (1980) Local Authority Airports: Accounts and Statistics 1979-80. CIPFA, London. Dillman, D. A. (2000) Mail and Internet Surveys: the tailored design method. 2nd Edition, John Wiley and Sons Inc, New York. Dobbie, L. (2001) The Control of Emissions Finding Solutions, Market Based Options. Paper presented to the 2nd Annual Aviation and the Environment Conference. Doganis, R. (1978) Airport Economics in the Seventies. Research Report No.5, Transport Studies Group, Polytechnic of Central London, London. Doganis, R. (1983) Economics of European Airports. Research Report No.9, Transport Studies Group, Polytechnic of Central London. Doganis, R. (1992) The Airport Business. Routledge, London. Doganis, R. and Graham, A. (1987) Airport Management: The role of performance indicators. Transport Studies Group, Polytechnic of Central London.

25

Favotto, I. (2001) Airport Retail Study.2000/01 edition. Arthur Andersen Airport Studies; Sydney, Australia. Francis, G. A. J. and Minchington, C. (2000) Value Based Metrics as Divisional Performance Measures, in Arnold, G. and Davies, M., (eds) Value-Based Management: Context and Application Wiley, Chichester, pp 151-162. Gosling, G. (1999) Aviation System Performance Measures. Nextor, Working Paper UCB-ITS-WP-99-1, Berkeley, California. Gosling, G. (2000) Aviation System Performance Measures for State Transportation Planning. Transportation Research Board 1703, pp. 7-16. Graham, A. (1991) Measuring Airport Performance. Paper presented at Airport Economics and Finance Symposium, Transport Studies Group, Polytechnic of Central London. Graham, A. (1999) Benchmarking Airport Economic Performance. Paper presented at the Airport Finance Symposium, Cranfield University, Cranfield. Graham, A. (2001) Performance Indicators for Airports. Business Management for Airports Short Course presentation, Loughborough University, Loughborough. Hegendorfer, H. and Morris, P. (2000) Competition Hots up for Top Airport Ratings. Airport World, Vol. 5, No. 3, pp. 26-29. Holloway, J., Hinton, M., Francis, G. and Mayle, D. (1999) Identifying Best Practice in Benchmarking, CIMA, London. Hulme, K., Gregg, M., Thomas, C. and Terranova, D. (2001) A Pilot Study into Complaints Caused by Aircraft Operations: Noise level and time of day. Paper presented at the 1st International Conference Environmental Capacity at Airports, Manchester Metropolitan University, Manchester. Humphreys, I. and Francis, G. A. J. (2000a) A Critical Perspective on Traditional Airport Performance Indicators. Proceedings of the 79th Annual General Meeting of the Transportation Research Board, Washington D.C.. Humphreys, I. and Francis, G. A. J. (2000b) A Critical Perspective on Traditional Airport Performance Indicators. Transportation Research Board 1703, pp. 2430.

26

Humphreys, I. and Francis, G. A. J. (2000c) Discussion of Gosling, G Aviation System Performance Measures for State Transportation Planning. Transportation Research Board 1703, pp. 14-15. Humphreys, I. and Francis, G. A. J. (2000d) The Past Present and Future Performance Measurement of Airports: The impact of changing ownership patterns. In Neely, A Performance Measurement 2000 Past Present and Future. Cranfield University, Cranfield, pp.251-258. Humphreys, I. Francis, G. A. J. and Fry, J. (2001) What are the lessons from Airport Privatisation, Commercialization and Regulation in the UK? Proceedings of the 80th Annual General Meeting of the Transportation Research Board, Washington D.C. IATA (2001) Global Airport Monitor. Aviation Information and Research Department, International Air Transport Association, London. ICAO (2000) Air Traffic Forecasts. http://www.icao.org/. Accessed on 21st May, 2001. Kazda, A. and Caves, R. (2000) Airport Design and Operation. Pergamon, Oxford. Muller, C. and Gosling, G. (1991) A Framework for Evaluating Level of Service for Airport Terminals. Transportation Planning and Technology, Vol. 16, pp. 3-28. Upham, P. and Yang, Y. (2001) A Scaled Approach to Developing a Decision Support System for Sustainable Airport Development. Paper presented at the 1st International Conference Environmental Capacity at Airports, Manchester Metropolitan University, Manchester.

27

Appendix 1: Worlds survey questionnaire and covering letters

Note: Percentage of respondents given for each closed question.

Airport Performance Measurement

This questionnaire forms part of a major project being conducted by Loughborough University and The Open University Business School on the use of performance measures in airports.

Q1

What job function do you work within? 9% Planning 30% Operations 40% Other (please specify) 14% Finance 7% Concession management

Q2

Is your organisation 19% Privately owned? 67% Government owned? Other (please specify) 14% Part Private - Part Government owned?

Q3

How many passengers did your airport handle last year? 29% 1 to 5 million 24% 5 to 10 million 23% 10 to 20 million 24% More than 20 million

Q4

Has your organisation employed any of the following methodologies to help improve performance? Please tick all that apply. 41% 36% 23% 12% 5% 27% 23% 25% 9% 46% 14% 12% Total Quality Management (TQM) Activity Based Costing Business Process Reengineering Business Excellence Model / European Foundation for Quality Management (EFQM) Malcolm Baldridge Award Environmental Management Systems (eg ISO14000) Quality Management Systems (eg ISO9000/BS5750or similar) Balanced Scorecard Value Based Management Best Practice Benchmarking No measures used Other (please specify)

Please turn over For internal use only

28

Q5

Is your organisation involved in any form of benchmarking? 72% Yes 23% No 5% Don't know

If yes, how would you describe your benchmarking experience?

....................................................................................................................................................

Please locate your organisation's experience of benchmarking on each of the following scales: concerned with specific tasks more to do with process improvement mainly internal comparisons primarily financial measures using mainly similar partners 5% 3% 8% 8% 14% 16% 5% 24% 21% 40% 16% 19% 8% 21% 24% 22% 8% 21% 32% 16% 24% 19% 10.5% 3% 3% 14% 38% 18% 10% 3% concerned with general practices 8% more to do with measurement 10.5% mainly external comparisons 5% primarily non-financial measures 0% using mainly dissimilar partners 3%

What have been the outcomes from your organisation's benchmarking?

....................................................................................................................................................

Q6

Which of the following performance measures does your organisation use and please rate the ones you do use on a scale of not useful (1) to very useful (5).

8.

Financial Yes 85% 85% 76% 77% 78% 61% 23% 32% 73% 32% 73% No 13% 13% 20% 19% 22% 37% 72% 61% 21% 63% 27% Dont know 2% 2% 4% 4% 0% 2% 5% 7% 6% 5% 0%

Income per passenger Expenditure per passenger Aeronautical income per passenger Non aeronautical revenue per passenger 2 Concession revenue per m Income per employee Value added per employee Capital expenditure per passenger Revenue to expenditure ratio Total revenue per Work Load Unit Other (please specify)

Not useful 1 0% 2% 6% 0% 3% 0% 0% 15% 0% 0% 0%

2 0% 10% 6% 0% 3% 15% 11% 15% 9% 8% 0%

3 24% 20% 15% 18% 15% 35% 11% 31% 31% 46% 0%

4 34% 33% 26% 35% 44% 19% 45% 8% 19% 23% 20%

Very useful 5 42% 35% 47% 47% 35% 31% 33% 31% 41% 23% 80%

29

Quality of service within airport terminal for the passenger Dont No know 22% 0% 31% 0% 39% 6% 36% 2% 24% 0% 17% 0% Not useful 1 2 0% 0% 0% 0% 0% 0% 0% 10% 0% 5% 0% 0% Very useful 5 50% 44% 54% 50% 62% 82%

Baggage delivery time Check in waiting time Availability of trolleys Walking distances Quality of signage / ease of finding way Other (please specify)

Yes 78% 69% 55% 62% 76% 83%

3 5% 9% 23% 20% 5% 0%

4 45% 47% 23% 20% 28% 18%

Environmental indicators Yes No 53% 43% 67% 28% Dont know 4% 5%

Percentage of passenger trips to the airport by public transport Percentage of homes subjected to a specified noise level Percentage of waste recycled per annum Number of community complaints about operations Other (please specify)

Not useful 1 0% 3% 0% 0% 0%

2 8% 0% 4% 7% 0%

3 4% 12% 22% 10% 0%

Very useful 4 5 21% 37% 30% 30% 35% 29% 55% 44% 48% 71%

50% 39% 11% 84% 13% 3% 86% 7% 7%

Q7

What performance measurement information is particularly useful in your own role?

....................................................................................................................................................

Q8

What additional performance measures would you like to help you in your job?

....................................................................................................................................................

Q9

If you personally use environmental indicators, please specify what they are

....................................................................................................................................................

Q10

If you personally use quality of service indicators, please specify what they are

....................................................................................................................................................

Please turn over

30

Q11

So far as you are aware, has your organisation introduced any new performance measures in the last two years? 58% Yes If yes, please list them below 42% No 0% Don't know

....................................................................................................................................................

How influential were management consultants in the choice of the new measure(s)? 23.5% Very influential 30% Some influence 21% Not influential 26.5% Management consultants not used

What other factors influenced the choice of the new measures?

....................................................................................................................................................

What tangible benefits has your organisation derived from this experience?

....................................................................................................................................................

Q12

What process do you use to collect performance measurement data?

....................................................................................................................................................

Q13

Do you use any comparative performance measurement data? 72% Yes 24% No 4% Don't know

If yes, what organisations do you compare yourself with?

....................................................................................................................................................

and what measures are compared?

....................................................................................................................................................

Q14

How are performance measures used to improve performance of your organisation?

....................................................................................................................................................

31

Q15

Who decides what performance measures to use?

....................................................................................................................................................

Q16

How is your organisation's use of performance measures monitored?

....................................................................................................................................................

Q17

From your perspective, what are the 5 most useful measures you work to? 1. 2. 3. 4. 5.

Q18

If your organisation uses a Balanced Scorecard, what are you views on this approach?

....................................................................................................................................................

Q19

Does your organisation have performance related pay? 56% Yes 44% No

If so, which performance measures is this linked to?

....................................................................................................................................................

and whom does it apply to? i.e. which role or grade

....................................................................................................................................................

Q20

What financial performance measures does your organisation use?

....................................................................................................................................................

Please turn over

32

Q21

The following measures form financial objectives for many organisations. Please select a maximum of five which are important to your organisation and rank them in order of importance (1-5) with 1 being the most important. 5% 3% 5% 3% 0% 3%
* *

Share price Earnings per Share P/E ratio Sales Return on sales Other (please specify)

13% 33% 0% 23% 13%

Return on Capital Employed Profit Stock levels Costs Cash flow

Percentage of respondents who gave each financial objective a rank of 1

Q22

As far as you are aware, which of the following financial measures are: used currently to evaluate the performance of your organisation being considered for use in the future for your organisation not being considered a measure you are not aware of Being Not being considered considered Not aware 0% 0% 2% 2% 14% 2% 10% 2% 4% 23% 38% 16% 12.5% 52.5% 22.5% 14% 49% 20% 9.5% 30% 9.5% 19% 57% 14% 18% 46% 21% 0% 75% 0%

Ability to stay within budget Profit Cash Flow Balanced Scorecard Economic Value Added (EVA ) Residual Income (RI) Return on Capital Employed (ROCE or ROI) Shareholder Value Added (SVA) Value drivers Other (please specify)

Used 98% 82% 84% 23% 12.5% 17% 51% 10% 15% 25%

Q23

Would you be interested in participating further in this study? 42% Yes 58% No

If yes, please give details to enable us to contact you: Name Job Title Organisation Address

OR ATTACH BUSINESS CARD

E-mail Telephone number

Thank you for taking the time to complete this questionnaire. Please return to Graham Francis, The Open University Business School, Walton Hall, Milton Keynes, UK in the international reply paid envelope provided

33

COVERING LETTER TITLE FIRSTNAME LASTNAME JOB_TITLE AIRPORT ADDRESS1 ADDRESS2 STATECITY COUNTRY 15th August 2000 Dear TITLE LASTNAME Performance Measurement in Airports We are writing to ask for your help in a study being conducted by The Open University Business School and Loughborough University. The purpose of the survey is to contribute towards a clearer picture of the use of performance measures in the top 200 of the world's busiest airports and to ascertain how they are changing over time. We realise that those working within the industry provide a unique and valuable view of what is actually happening. In order to keep our teaching and research current we are keen to receive your contribution. We should be grateful if you would complete the questionnaire as far as possible. Please answer all the questions which you are able to, if there are some questions you are unable to answer please leave them blank but do not let this stop you from completing the other questions. The information you provide will be treated in the strictest confidence and will only be used for academic purposes. Neither individuals nor companies will be identified in any analysis. If you have any questions or comments about this study, we would be happy to answer them. Please contact Graham Francis or Dr. Ian Humphreys by email or phone. Alternatively write to Graham Francis at the address above. Thank you very much for helping with this survey. Yours sincerely

Graham Francis BSc MA MBA ACMA Performance Management Research Unit The Open University Business School G.A.J.Francis@open.ac.uk + 44 1908 858265

Dr. Ian Humphreys BA MSc PhD MCIT Transport Studies Group Loughborough University I.M.Humphreys@lboro.ac.uk + 44 1509 223422

34

REMINDER LETTER 1

TITLE FIRSTNAME LASTNAME JOB_TITLE AIRPORT ADDRESS1 ADDRESS2 STATECITY COUNTRY 26th September 2000

Dear TITLE LASTNAME Performance Measurement in Airports Six weeks ago we sent you a questionnaire seeking your opinions on performance measurement in airports. You were selected because you work at one of the worlds top 200 busiest airports. If you have already completed and returned the questionnaire to us, please accept our sincere thanks. If not, we would be grateful if you could complete the enclosed replacement questionnaire as soon as possible. We are especially grateful for your help because it is only by asking people like you to share your experiences that we can understand more about the use of performance measures at airports and reflect this in our teaching and research. The information you provide will be treated in the strictest confidence. Neither individuals nor companies will be identified in any analysis. Thank you very much for helping with this survey. Yours sincerely

Graham Francis BSc MA MBA ACMA Performance Management Research Unit The Open University Business School G.A.J.Francis@open.ac.uk + 44 1908 858265

Dr Ian Humphreys BA MSc PhD MCIT Transport Studies Group Loughborough University I.M.Humphreys@lboro.ac.uk + 44 1509 223422

35

REMINDER LETTER 2

TITLE FIRSTNAME LASTNAME JOB_TITLE AIRPORT ADDRESS1 ADDRESS2 STATECITY COUNTRY 7th November 2000 Dear TITLE LASTNAME Performance Measurement in Airports On the 15th August we sent you a questionnaire seeking your experiences and opinions on performance measurement in airports. To the best of our knowledge, it has not yet been returned. The comments from people who have already responded include a diversity of reasons for measuring airport performance and some interesting points regarding the effectiveness of different measures. Many have described their experiences, both good and bad, of working towards different performance measures. We think the results will aid our understanding and will contribute to the improvement of performance measurement within the industry. We are writing to you again because of the importance your opinions have for helping to get accurate results. It is only by hearing from nearly everyone in the sample that we can be sure the results are truly representative. The information you provide will be treated in the strictest confidence. Neither individuals nor companies will be identified in any analysis. Protecting the confidentiality of peoples answers is very important to us, as well as to the universities. We hope you will fill in and return the questionnaire as soon as possible, but if for any reason you prefer not to answer it, please let us know by returning a note or blank questionnaire in the enclosed international reply paid envelope. Thank you very much for helping with this survey. Yours sincerely

Graham Francis BSc MA MBA ACMA Performance Management Research Unit The Open University Business School G.A.J.Francis@open.ac.uk + 44 1908 858265 36

Dr Ian Humphreys BA MSc PhD MCIT Transport Studies Group Loughborough University I.M.Humphreys@lboro.ac.uk + 44 1509 223422

37