P j t Completion

Project C l ti Reports
R t
(PCRs)
( )

Gender Specialists’
Specialists Annual Workshop
and Retreat
Tuesday October 26 2010
Tuesday,

The views expressed in this paper are the views of the author and do not necessarily reflect the views or policies of the Asian
Development Bank (ADB), or its Board of Governors, or the governments they represent. ADB does not guarantee the accuracy of
the data included in this paper and accepts no responsibility for any consequence of their use. The countries listed in this paper do
not imply any view on ADB's part as to sovereignty or independent status or necessarily conform to ADB's terminology.

1
Tuesday, 26 October 2010 IED
 Acknowledgements
Presentation draws upon the works and
presentations of Director-General, Directors,
and other IED staff

2
Tuesday, 26 October 2010 IED
 S
Section
i I

3
Tuesday, 26 October 2010 IED
 IED Evaluation of ADB’s
Support to GAD in Two Phases
Phase I: Meta-Evaluation
Meta Evaluation
- Criteria: relevance, responsiveness, results
- Evaluation completed in Dec 2009
- Favorable response to IED’s
IED s three strategic
recommendations

Tuesday, 26 October 2010 IED 4

 Management Response
…To
To follow up on IED
IED'ss recommendations
recommendations, we
intend to introduce better monitoring
mechanisms to track gender equality results
of projects throughout implementation…
implementation

International Women's Dayy Breakfast,, Keynote

y Address byy
President Kuroda, 8 March 2010

Tuesday, 26 October 2010 IED 5

 Phase I – Recommendations
Recommendation 1: Improve outcome orientation of
GAD goals and their monitoring and evaluation.
• Define expected outcomes from gender
mainstreaming
i t i and d sett targets
t t ffor outcomes.
t
• Explicitly include baseline gender data in all gender
assessments and DMFs
DMFs. Also include GAD targets
particularly on outcomes in DMFs.
• Include reporting
p g on GAD
GAD--related achievements
(outputs, outcomes) in monitoring and evaluation
activities and related reports (PPRs, PCRs, PPERs,
SAPEs CAPEs)
SAPEs, CAPEs).
Tuesday, 26 October 2010 IED 6
 Phase I - Recommendations
Recommendation 2: Provide clarity to operationalize
GAD goals in ADB operations in the context of
Strategy 2020
• To achieve the intended objectives of gender
mainstreaming, clarify and examine approaches and
targets to adopt, given the realities of sector
orientation in the Strategy 2020 and gender capacity
constraints faced by operations departments.
• In particular, define the approaches for gender
mainstreaming in private sector
operations/nonsovereign operations, large
infrastructure development and policy-
policy-based
p
operations,, and provide
p specific
p directions to staff.

Tuesday, 26 October 2010 IED 7

 Phase I - Recommendations
Recommendation 3: Provide adequate financial and
human resources to support policy implementation.
• Following the actions in recommendations 1 and 2
above assess and ensure adequate staffing and skill
above,
levels at headquarters and RMs required for
effectively implementing the Policy.
• Recognize best performing project teams and/or
departments in achieving gender mainstreaming and
gender outcomes.

Tuesday, 26 October 2010 IED 8

IED Evaluation of ADB’s Support
to GAD in Two Phases
Phase II: Country Case Studies
- 6 countries selected
- Country perspective,
perspective project level information,
information
gender results on the ground
- 5 step assessment approach inorder to ensure
comparability
- Scheduled for December 2010

9
 S
Section
i II

Tuesday, 26 October 2010 IED 10

 Purpose of Evaluation
• Accountability: Whether ADB is doing right things,
resources are properly allocated and used
used,
intended outcomes realized
• Better knowledge management: Identify lessons
to improve development impact of future strategy
and operations
• Development
D l t effectiveness:
ff ti M i i
Maximize
development effectiveness of ADB operations by
evaluation feedback

Tuesday, 26 October 2010 IED 11

 Who Evaluates What?
Policies, Business Processes,
Country, Sector, Project
Development

R
Results
lt
Design and Monitoring
Framework

Project/Program Real-time/Project Post

Monitoring
Design Completion Completion
Implementation Reporting
Evaluation Evaluation

OD/IED OD OD/IED IED

Tuesday, 26 October 2010 IED 12

 PCRs – Objectives
• Learn from the experience of Borrowers
Borrowers, EAs
and ADB
• U
Understand
d t d what h t worked,
k d what
h t did not,
t and
d
their underlying reasons
• Identify lessons to improve design of new
projects

Tuesday, 26 October 2010 IED 13

 PCRs - Scope
• Assessment of project design and implementation
plan
• Evaluation of - Project preparation
preparation, appraisal
appraisal,
implementation arrangements; Performance of
Borrowers, EAs, ADB; Operations and likelihood of
sustainability (Preliminary); Achievement of project
outcomes and expected impact (Preliminary)
• R
Recommendations
d ti to
t improve
i future
f t projects
j t –
design, implementation and operations

Tuesday, 26 October 2010 IED 14

 PCRs – Target audience
• ADB’s Board of Directors
• Management, Vice-Presidents and Head of
D
Departments
t t
• Country Directors, and RMs
• Borrowers, EAs and IAs
• Online Dissemination

Tuesday, 26 October 2010 IED 15

Common attributes of good PCRs
• Focus on achievement of objectives,
objectives not
necessarily production of outputs or
fulfillment of conditions
• Straight-forward attribution
• Clarity
• Internal consistency
• Adequate evidence to support claimed
‘ratings’
g
Tuesday, 26 October 2010 IED 16
 PCR Ratings-
Ratings- Sovereign
Projects
Non-Core Criteria:
Core Criteria:
• Impacts:
• Relevance
• Socioeconomic
• Effectiveness
• Environment
• Efficiency
• Institutional
• Sustainability
S t i bilit • ADB’s performance
• Executing
g Agencies’
g
performance
Tuesday, 26 October 2010 IED 17
 Relevance ((20
20%)
%)
Consistency of a project’s impact and outcome with
• Government
Government’ss development strategy
• ADB’s lending strategy for the country
• ADB
ADB’ss strategic objectives at the time of
approval and evaluation
• Adequacy of the design

Tuesday, 26 October 2010 IED 18

 Relevance criteria and GAD –
questions used in Phase II study
• Do the CGA , CPS (and its gender strategy)
identify the sector/project as priority for
gender-related interventions?
• How important is gender to achievement of
overall project objectives, as articulated in
project
p j documents?
• Is there evidence of gender-inclusive
consultations and participation (C&P) on
project
j tddesign
i and d intended
i t d d outcomes?
t ?
• How well are the findings of the project
gender analysis (or gender focus of social
analysis) reflected in design of the project?
Tuesday, 26 October 2010 IED 19
 Effectiveness (30
(30%)
%)

The extent
Th t t tto which
hi h th
the outcomes,
t as specified
ifi d in
i
the design and monitoring framework, either as
agreed
ag d at
a approval
app o a oro as subsequently
subs qu y modified,
od d,
have been achieved.

Tuesday, 26 October 2010 IED 20

Effectiveness criteria and GAD –
questions used in Phase II study
• Are gender-related outcomes and/or outputs
included in the DMF, and are there associated
targets and indicators?
• Are there missed opportunities for
mainstreaming gender in project design?
• Are
A e gender-related
gende elated design features
feat es (acti
(activities)
ities)
well conceived, and appropriately reflected in
the Gender Action Plan (GAP)?
• Were gender-related outputs delivered as
planned and on time, or, for active projects,
are they considered “on
on track”?
track ?
Tuesday, 26 October 2010 IED 21
 Efficiency
y (30
(30%)
%))
Describes (ex-post) how economically resources have
been converted into results
results, using
• the EIRR (relative to 12% standard)
• cost-effectiveness
cost effectiveness of the investment (relative to
industry practice)
• other indicators (implementation issues)

Describes the resilience to risk of net benefit flows

over time

Tuesday, 26 October 2010 IED 22

 Efficiency
y criteria and GAD –
questions used in Phase II study
• Were sufficient human and financial
resources allocated to gender-related
activities in the project?
• Are gender activities or indicators monitored
during supervision?
• Has the project
p oject complied with
ith all gende
gender-
related covenants and agreements, or is it on
track to do so?
• Does the Gender Action Plan (GAP) provide a
suitable foundation for monitoring
implementation of gender-related activities?
Tuesday, 26 October 2010 IED 23
 S t i bilit (20
Sustainability (20%)
%)
Considers the likelihood that human
human,
institutional, financial, and other resources
are sufficient to maintain the outcome over
the economic life of the project

Tuesday, 26 October 2010 IED 24

Sustainability
y criteria and GAD –
questions used in Phase II study
• Does the project build longer-term country
institutional capacity to address gender issues
in the project or sector?
• Is there continued demand for project-
provided gender-related products and services?
• Have
Ha e project-supported
p oject s ppo ted policy,
polic legal
legal, and
institutional reforms been implemented or are
policy and institutional arrangements in place
t sustain
to t i gender-related
d l t d activities?
ti iti ?
• Are there gender-related risks to achievement
of project outcomes? (Note: scale is inverted--
high risk = 0; low risk = 3)?
Tuesday, 26 October 2010 IED 25
 Assessment Ratingg - Categories
g
and Ranges for four criteria

Highly successful (HS>= 2.7)

Successful (1
(1.6=<S<2.7)
6=<S<2 7)
Partly successful (0.8=<PS<1.6)
Unsuccessful (U<0.8)

Tuesday, 26 October 2010 IED 26

 PCR – Length and contents
• About 15 pages of the main text
• PCR should focus on the analysis of project
inputs activities,
inputs, activities costs
costs, implementation and
institutional arrangements, outputs,
outcomes, and impacts.
• Preliminary assessment of project
sustainability

Tuesday, 26 October 2010 IED 27

 Cancelled projects
For cancelled projects without any
disbursements, PCRs need not be prepared;
instead a Board Information Paper should be
prepared to spell-out reasons for
cancellations

Tuesday, 26 October 2010 IED 28

 Project Performance
Evaluation Reports – PPER
Guidelines
IED’s PPER Guidelines and the definition of
rating categories, rating criteria, etc., should
be followed for the preparation of PCR

Tuesday, 26 October 2010 IED 29