Measuring performance, benchmarking

and setting objectives

ATC Global
Session 3

Ralph RIEDLE
Chairman Performance Review Commission
17 September 2014

Topics

ANS in the European aviation context

Performance oriented approach in ANS
ANS performance review examples
Conclusions

Measuring performance, benchmarking and setting objectives

ANS in the European aviation context (1/2)

GDP from Air Transport
in EU $160 B
Source: ATAG

Air Navigation Services

$10 B

Safety is the primary objective!

No accident with ANS contribution since 2011
Reported incidents in 0.3% of flights

Safety
European
IFR traffic

Environment

Costefficiency

6% of Airline
operating costs (Europe)

Capacity

6% of aviation related CO2

emissions (0.2% of total emissions)

Air transport delay (2013)

All 9 min. per flight
ANS-related 1 min. per flight

Although ANS is comparatively small in aviation context.

Measuring performance, benchmarking and setting objectives

ANS in the European aviation context (2/2)

the stakes are high!
ANS generates.
Value..
Safety
Efficient flow of air traffic

SES area
Indirect
service
quality
related
costs

$4 B

Costs .

Environmental impact
High penalties to economy if disrupted
Scope for improvements!

efficiency

ANS related
delays
ATCO

Total economic cost $ 14 B (SES area)

Direct ANS provision costs (user charges)
Indirect service quality related costs
(delays, non-optimum flight profiles)
Airborne equipment costs to be added

Flight

costs

Total
economic
cost

$14 B
Direct
ANS
Cost

Support
costs

$10 B

Measuring performance, benchmarking and setting objectives

CAPEX

Other costs

Performance-oriented approach in ANS (1/5)

You cant manage what

you dont measure!
Measuring performance, benchmarking and setting objectives

Performance-oriented approach in ANS (2/5)

Strategy &
objectives
(what must be done well to implement strategy )

Verified
Data

Reporting,
monitoring
& analysis
(actual vs. targets)
(understand achieved
performance)

Performance
Review

Framework,
KPIs & PIs
(How success
should be
measured)

Target setting
(Quantification of what should be achieved)

Objectives (What should be achieved?)

Commonly accepted framework (What should be measured, targeted: KPAs, KPIs?)
Performance targets (How much should be achieved? When?)
Performance monitoring (Are we on track?)
Corrective measures (As necessary)
Measuring performance, benchmarking and setting objectives

Performance-oriented approach in ANS (3/5)

Delay
crisis

1990

Delay
crisis

1997

1998

2004

PRC

SES

ECAC Common
ATM Performance
strategy

2009

2010

SES II PRB

2012

Start
RP1

2015

Start
RP2

ECAC institutional strategy (1997)

Common ATM Performance strategy

Focus on outcome, not only means (e.g. technology)

Independent Performance Review Commission (PRC) (1998)

Independent Performance review
Light-handed regulation: objective information, recommendations

Single European Sky (SES II: 2009)

SES Performance scheme

Enforceable performance targets (EU, national/FAB levels), incentives
Performance Review Body (PRB) advising the European Commission
Measuring performance, benchmarking and setting objectives

Performance-oriented approach in ANS (4/5)

ICAO
high-level expectations of the ATM community
Safety
Security

Safety
management

Participation

Costeffectiveness

Access &
Equity

Cost
management

European focus
areas for target
setting

Interoperability

Capacity
management

Environmental
sustainability

Capacity

Predictability

Flexibility
Efficiency

Environment

European focus areas in line with the 11 Key performance areas (KPAs)
defined in ICAO Global Air Traffic Management Operational Concept
Binding targets for Safety, Cost-efficiency, Capacity and Environment

Other KPAs measured

Measuring performance, benchmarking and setting objectives

Performance-oriented approach in ANS (5/5)

Within
region
Safety
management

Cost
management

European focus
areas for target
setting
Environmental
sustainability

Capacity
management

Benchmarking dimensions

Over
Time

Across
regions

Different dimensions of performance benchmarking enable to:

Understand and position own performance
Identify performance gaps and scope for improvement
set ambitious but realistic targets
Well established publications on ANS performance in Europe:
Annual Performance Review Report (system view, all KPAs)
ATM Cost Effectiveness Benchmarking (ACE) reports (Economics)
Ad hoc reports (US/ Europe comparisons, etc.)
Measuring performance, benchmarking and setting objectives

Annual Performance Review Reports (PRR)

Capacity

SAFETY: Review of safety incident evolution

in order to identify trends (RI, SMI, etc.)

PRU analysis

Flight Plan (KEP)

Actual trajectory (KEA)

ENVIRONMENT: Monitoring of flight

efficiency en route and in terminal areas;

4,5

2,0

0,63

100

0,53

1,1

1,4
0,9

1,2

1,0

0,9

0,9

0,8

1,4

110

90

Traffic index (base: 1997)

2,5

2,9

120

2,2

2,9

130

80

ATC Other (strike, equipment, etc.)

WEATHER

OTHER (Special event, military, etc.)

IFR Traffic

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

70

2000

2013

2012

2011

2,0

1999

2009/10 KEA data based

on regression analysis

150

140

1998

3,0

5,0
4,5
4,0
3,5
3,0
2,5
2,0
1,5
1,0
0,5
0,0

1997

3,5

3,14

3,31

4,0

3,20

4,5

En-route ATFM delay/ flight (min.)

4,86

5,0

4,87

4,91

5,5

2,5

COST-EFFICIENCY: Analysis of ANS unit costs

und underlying drivers
Average en-route ATFM delay per flight

6,0

2010

Environment

2009

European
IFR traffic

Costefficiency

inefficiency (%)

Safety

source: Network Manager

CAPACITY: Evaluation of ANS related en route

and airport delays and constraining factors;

Verified facts and performance indicators

Independent critical analysis (wide spectrum)
Recommendations to decisions makers
Measuring performance, benchmarking and setting objectives

10

ANSP benchmarking ACE reports

Safety
European
IFR traffic

Environment

Costefficiency

Capacity

ATM Cost Effectiveness Benchmarking (ACE) reports

Widely accepted performance framework (ICAO Doc 9161)

Verified data, official performance indicators and metrics
Factual analysis at European level, within region, over time
Purpose: Inform decision makers, support target setting within SES, etc
Measuring performance, benchmarking and setting objectives

11

Global perspective: US-Europe comparison

2012/2013

US vs.
Europe

Europe

USA

Geographic Area (million km )

Nr. of civil en route Air Navigation Service Providers

11.5
37

10.4
1

-10%

Number of en route centres

Number of APP units (Europe) and terminal facilities (US)

63
260

20
163

-43
-97

Number of fully trained Air Traffic Controllers (ATCOs in Ops.)

Total staff
Controlled flights (IFR) (million)

17 200
58 000
9.6

13 400
35 500
15.1

-22%
-39%
+57%

Flight hours controlled (million)

Average length of flight (within respective airspace)

14.3
551 NM

22.4
515 NM

+57%
-7%

1.2
3.9%

2.2
21%

x1.7

Relative density (flight hours per km )

Share of General Aviation

Facts

Similar conditions (geographical area, average flight length)

US controls more IFR traffic (+57%) with fewer staff (-39%),

including ATC Controllers (-22%)

What are the underlying drivers of performance?

Measuring performance, benchmarking and setting objectives

12

Performance in capacity and cost-efficiency

En route
delay

Traffic index

Reactive policy in the 90s: delays go up while costs go down, and vice versa
As of 1998, performance-oriented approach and improved capacity management
contributed to reduction of both delays and unit costs
Enforceable SES performance targets apply from 2012 onwards
Measuring performance, benchmarking and setting objectives

13

Environmental performance
6.0

5.42

route extension (%)

5.0

5.38

5.18

5.15

5.11

3.17

3.12

Filed flight plan

4.0
3.0

Actual flown
trajectory

2.0

3.29

ENVIRONMENT:
Performance targets on
en route flight efficiency
within SES scheme

1.0
Source: PRU analysis

0.0

2009

2010

2011

2012

2013

2014

2015

2016

2017

Target RP1

2019

Targets RP2

Flight efficiency first measured in 2006: continuous improvement since then

Good routing efficiency of ANS (3%) compared to other transport modes

2018

Yet significant economic impact (fuel burn, flight time)

Impossible to reach 0% with full civil-military traffic load

SES targets on Environment set for 2014 (FPL), 2019 (Actual, FPL)

Improved flight-efficiency compensates for air traffic growth

Carbon-neutral growth of aviation (due in 2020)

already being met as far as European ANS is concerned!
Measuring performance, benchmarking and setting objectives

14

Conclusions

Performance is the bottom line for ANS policy

Impact on more than $14B p.a. in EU, while ensuring safety

European Performance-driven strategy delivers,

but margins for significant further improvements remain
Independent Performance review since 1998 (PRC)
Stronger regulation under SES II (enforceable targets, regulations)
EUROCONTROL supports both PRC and EC/PRB

Global benchmarking reveals best practices, weaknesses

EUROCONTROL willing to engage with benchmarking partners

More details in workshop Driving excellence in ATM

performance ( Workshop theatre, 19 Sep. , 10am)
Reports available at http://www.eurocontrol.int/prc/publications
Measuring performance, benchmarking and setting objectives

15