STAR 2012 & 2013 Results

November 2014

Contents
Methodology
Portfolio Analyzed
Results
Analysis
Portfolio Outlook until 2020

Methodology

STAR Principles
Sustainable Transport Appraisal Rating
Measures projects contribution to delivering economic,
social and environmental objectives
Rating
Sustainable transport objectives Score
7 to 10
Highly Sustainable
Accounts for risk
5 to 6
Sustainable
Partly guided, qualitative
3 to 4
Moderately Sustainable

Ratings:
Outputs:

1 to 2

Marginally Sustainable

-1 to 0

Moderately Unsustainable

-2 to -4

Unsustainable

-5 to -10

Highly Unsustainable

Aggregate rating of sustainability

Separate ratings by dimension of sustainability

STAR Construct
Economic
Risk to Sustainability

Design & evaluation risk

Implementation risk
Operational risk

1. Efficiency: people &

businesses
2. Quality & reliability
3. Fiscal burden
4. Employment
5. Wider economic benefits:
cross-border, urban, rural

Social
Environmental
12.GHG Emissions
13.Air pollution
14.Natural & built
environment
15.Resource efficiency
16.Climate resilience

6. Basic accessibility
7. Safety
8. Affordability
9. Inclusion
10.Social cohesion
11.Employment

Portfolio Rating Methodology

Evaluation choices:
Appraisal stage - based on public RRP documents
Projects approved in 2012 and 2013

Rating Process
Rating team:
One from each ADB transport division
One external auditor

Common score book

Two raters per project: ADB staff + auditor
Rater independence
Differences resolved in joint meeting
Specific guidance issued for safety and GHG

Portfolio Analyzed

Portfolio Analyzed: 46 Projects, $7Billion

By Subsector
2012

Projects %
$

%
Road Transport
15
63% 2,652,940 71%
Rail Transport
0
0%
0 0%
Water Transport
3
13%
195,800 5%
Air Transport
1
4%
6,920 0%
Urban Transport
5
21%
906,000 24%
Transport Mgt. & Policies
0
0%
0 0%
100% 3,761,660 100%
Total
24

Projects

Road Transport

1
8

Rail Transport

Water Transport

Air Transport

Projects
15
2
0
1
3
1
22

30

Urban Transport
Transport Management
and Policies

2013
%
68%
9%
0%
5%
14%
5%
100%
6%

$
2,064,200
280,000
0
130,000
390,200
400,000
3,264,400

Share of Lending

18%
2%
3%
4%

%
63%
9%
0%
4%
12%
12%
100%

67%

Portfolio Analyzed: 46 Projects, $7Billion

By Geographic Region
2012
Projects %
$
9
1,512,300
38%
3
311,400
13%
4
87,940
17%
7
1,350,020
29%
1
500,000
4%
24 100% 3,761,660

Central-West
East Asia
Pacic
South Asia
Southeast Asia
Total

Projects

%
40%
8%
2%
36%
13%
100%

Projects
5
4
3
7
3
22

2013
%
23%
18%
14%
32%
14%
100%

$
563,000
610,000
289,000
967,400
835,000
3,264,400

Share of Lending

4
14

CWRD

19%
30%

EARD
14

PARD
SARD
7
7

%
17%
19%
9%
30%
26%
100%

SERD

33%

13%
5%

Results

STAR Results
2012

2013

2 Years

Marginally
Marginally
Marginally
Moderately Sustainable
Moderately Sustainable
Moderately Sustainable
Sustainable
Sustainable
Sustainable
Sustainable or Above
Sustainable or Above
Sustainable or Above
or Below
or Below
or Below

By Share of Lending
By Number of Projects

39%
38%

41%
38%

20%
24%

37%
45%

48%
41%

15%
14%

38%
41%

44%
39%

2013

2012 and
2013

18%
20%

100%

100%

Highly Sustainable

80%

Sustainable

60%

80%
60%

Moderately Sustainable
40%

40%

Marginally Sustainable
20%

20%

Moderately Unsustainable
0%

0%

2012

Projects

2013

2012 and
2013

2012

Share of Lending

Sub-Ratings by Objective
12%

(2 years)

18%

31%
32%
72%

44%

Sustainable or Above
Moderately Sustainable

57%

Marginally Sustainable or
Below

56%
38%

26%
2%
Economic

12%
Social

Environment

Overall

Analysis

Ratings By Subsector: STAR

Moderately
Unsustainable

Marginally
Sustainable

Moderately
Sustainable

Sustainable

Highly Sustainable

Road Transport

Rail Transport

Water Transport

Air Transport

Urban Transport

Transport Mgt. & Policies

Ratings By Subsector: STAR

No

ROADS

URBAN

Trunk Road

25

57%

Rural Road

9%

Road Maintenance

1%

Ring Road

1%

BRT

7%

Metro

11%

4%

3%

2%

1
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

6%

Railway
Water
OTHER

%lending

Air Transport
Policy Loan

Marginally Sustainable or Less

Sustainable or Above

Moderately Sustainable

Ratings By Subsector: STAR

BRT
Water
Road Maintenance
Rural Road
Metro
Railway
Air Transport
Policy Loan
Ring Road
Trunk Road

Moderately Marginally
Unsustainable Sustainable

Moderately
Sustainable

Sustainable

Highly
Sustainable

Ratings By Lending Instrument

Moderately
Unsustainable

Marginally
Sustainable

Project Loan

Moderately
Sustainable

AddiXonal Financing

Sustainable

Sector Loan

MFF

Highly Sustainable

Policy Loan

Ratings By Country Classification

Moderately
Unsustainable

Marginally Sustainable

Moderately
Sustainable

Sustainable

Highly Sustainable

Rating: Regional Projects

Moderately
Unsustainable

Marginally Sustainable

Moderately
Sustainable

Regional

Sustainable

Non-Regional

Highly Sustainable

Other findings from ratings

Economics input in project selection & design limited
Great diversity of practices with regard to road safety
Rather successful mainstreaming of climate resilience
Highest rated projects (STAR):
Best: BRTs, water, road maintenance & rural roads,
Second best: railway/metro and selected trunk road projects
Middle and upper middle-income countries

Issues with information available

MFF information on 2nd and other tranches
RRP information on project content and impacts/risks
Specific impacts not understood: safety, climate change
Economic analysis weak points: demand, benefit analysis
IPSA on social risks/ benefits
Risk analysis focus on implementation risks, not
development risks

Lessons on rating process

Ratings subjective, but dispersion of results limited
Good consistence achievable through team rating
Often information not in RRP / has to be inferred
Time budget:
STAR desk-based rating process: 4-8 h/project
Initial training + team discussions needed to train raters
Basic self-rating by project officer: 10-30 min
Complete detailed report with indicators, justifications and
ratings possible as part of PPTA

Performance Outlook
until 2020

Performance Factors (1)

Portfolio composition:
2009-2013 actual
2014-2017 portfolio pipeline analysis
2018-2020 scenario to close gap with STI targets

Portfolio composition - ADB

100%
90%
30%

80%
70%
60%

25%

50%
40%

30%
20%

42%

10%
0%
2009-2011

2012-2014

2015-2017

2018-2020 ?

Trunk Road

Rural Road

Road Maintenance

Railway

Urban roads and trac management BRT

Metro

Air Transport

Water

STI Targets

Portfolio assumptions by RD
4,000

4,000

CWRD

3,500

3,500

3,000

3,000

2,500

2,500

2,000

2,000

1,500

1,500

1,000

1,000

EARD

500

500

-
2009-2011

2012-2014

Trunk Road

2015-2017

2018-2020?

Rural Road

2009-2011

2012-2014

2015-2017

Road Maintenance

Railway

Urban roads and trac management BRT

Metro

Air Transport

Water

2018-2020?

Portfolio assumptions by RD
4,500

4,500

SARD

4,000

4,000

3,500

3,500

3,000

3,000

2,500

2,500

2,000

2,000

1,500

1,500

1,000

1,000

500

500

-
2009-2011

2012-2014

2015-2017

2018-2020?

SERD

-
2009-2011

2012-2014

2015-2017

Trunk Road

Rural Road

Road Maintenance

Railway

Urban roads and trac management

BRT

Metro

Air Transport

Water

2018-2020?

Portfolio assumptions by RD
800

PARD

700

Water

600

Air Transport
500

Metro
BRT

400

Urban roads and trac management

Railway

300

Road Maintenance
Rural Road

200

Trunk Road
100

-
2009-2011

2012-2014

2015-2017

2018-2020?

Portfolio composition - ADB

100%
90%
30%

80%
70%
60%

25%

50%
40%
30%

42%

20%
10%
0%
2009-2011

2012-2014

2015-2017

2018-2020 ?

Trunk Road

Rural Road

Road Maintenance

Railway

Urban roads and trac management BRT

Metro

Air Transport

Water

STI Targets

STAR Performance Assumptions

2009-2011 : historic average ratings by subsector
2012-2013 : actuals
2014-2020: 2 scenarios
Historic average ratings by subsector
STI mainstreaming scenario: stronger action on maximizing
social and environmental positive impacts+ moderately better
economics

Outlook on sustainability

PorJolio change
only

2009-
2011

2012-201
4

Marginally sustainable and less

38%

34%

31%

26%

Moderately sustainable

54%

53%

48%

45%

Sustainable and beNer

8%

13%

21%

30%

2015-2017 2018-2020

Outlook With Mainstreaming

PorJolio change
only

PorJolio +
Mainstreaming

2009-
2011

2012-201
4

Marginally sustainable and less

38%

34%

31%

26%

22%

9%

Moderately sustainable

54%

53%

48%

45%

52%

50%

Sustainable and beNer

8%

13%

21%

30%

26%

41%

2015-2017 2018-2020 2015-2017 2018-2020

Scope for Change

8%

13%
26%
39%

54%

Sustainable and be^er

53%
52%

Moderately sustainable
51%

38%

34%
22%
10%

2009-2011

2012-2014

2015-2017

2018-2020?

Marginally sustainable and less

Scope for change

2009-2011

2018-2020

39%
38%

STI
10%
8%

Less
Sustainable

Marginally
Sustainable

Moderately
Sustainable

Sustainable

Highly
Sustainable

Less
Marginally Moderately Sustainable
Highly
Sustainable Sustainable Sustainable
Sustainable

Trunk Road

Rural Road

Road Maintenance

Railway

Urban roads and trac management BRT

Metro

Air Transport

Water

Conclusions and next steps

Portfolio composition is a strong driver of share of
sustainable projects
Mainstreaming is key to reduce tail
STI considerations:

Target for railway seems difficult to reach

Target for urban reachable? Requires 2.5x more lending
Road and urban targets could be split / revised
STAR portfolio targets?
Portfolio targets by department?

Conclusions and next steps

Proposed discussion by division of results and outlook
STAR options

Annual reporting on new projects and portfolio

Gradual introduction in parallel to informal peer review
Use of framework during PPTA on voluntary basis
Review of future project pipelines by country or region?

Thank you

Additional Slides

Portfolio composition assumptions

CWRD
EARD
PARD
SARD
SERD
Total

Road
Rail
Urban
Air
Water
Total

2009-2011
900
370
100
500
340
2,200
2009-2011
1,700
350
100
50
50
2,200

2012-2014
1,000
470
140
1,450
500
3,500
2012-2014
2,550
350
450
50
100
3,500

2015-2017
1,180
730
210
950
650
3,700
2015-2017
2,450
600
500
100
50
3,700

2018-2020?
1,200
450
250
1,200
900
4,000
2018-2020?
2,300
550
1,000
100
100
4,000

STI Target -
2020
1,650
1,000
1,200
50
100
4,000

Results profiles by sector

Strengths

Weaknesses

Trunk Roads Generally strong

economics
Simple
PosiXve safety impacts
when 4-laning
Climate resilience
increasingly
mainstream

Many projects have low economic

returns: overdesign, regional projects
NegaXve safety when widening, no
systemaXc approach to audit
Moderate negaXve impacts on
environment, limited improvements
possible
No consideraXon for transport services
Few projects address sector-wide issues
Maintenance risk

Rural roads

Economic benets may be high or low

depending on approach and area
No consideraXon for transport services,
user needs
Maintenance

Strong social benets

Generally posiXve
environmental
impacts

Results profiles by sector (2)

Strengths
Road
Very strong economics
maintenance Good potenXal for social
benets

Weaknesses
Safety impacts ocen negaXve if not
carefully miXgated

BRTs

Strong economics if at
ImplementaXon complexity
grade
Strong social/environment
(rarely major though)

Metro

Similar benets as BRT

projects, but less value-
for-money

Railways

Strong social/environment Economics/sustainability for

benets if modal shic
passenger railways
Strong economics if freight Risks that modal shic not achieved/
sustained

Simpler implementaXon than BRTs,

cost overruns

Results profiles by sector (3)

Comments
Water
transport

Inland water transport: environmental impacts may be mixed,

social impacts limited, economics depend on modal shic potenXal
Inter-island (remote areas): OK economics (esp. reliability), strong
social benets (safety, access), posiXve environment (emissions,
resilience)

Airports

Strong economics
No direct posiXve impacts on poverty/social
NegaXve on environment, especially GHG, but improved resilience

Ring road

Moderate posiXve economics

Strong negaXve on social, environment

Railways

Strong social/environment benets if modal shic

Strong economics if freight
Economics/sustainability for passenger railways
Risks that modal shic not achieved/sustained

STAR Scoring: Step 1

Step 2

STAR Scoring: Step 3

Rating

Highly
Unsustainable

Score

-5 to -10 -2 to -4

Unsustainable

Moderately
Unsustainable

Marginally
Sustainable

Moderately
Sustainable

Sustainable

Highly
Sustainable

-1 to 0

1 to 2

3 to 4

5 to 6

7 to 10