Use of PEFA assessments

ICGFM Conference
Miami – May 21, 2009

PEFA Secretariat
 What can countries use the
PEFA Framework for?

 Harmonize information needs

• Common information pool
 Inform PFM reform formulation
and priorities
 Compare to and learn from peers
 Monitor results of reform
efforts
2
 What can countries use the
PEFA Framework for?

 Harmonize information needs

 Inform PFM reform formulation
and priorities
 Compare to and learn from peers
 Monitor results of reform
efforts

3
 PEFA reports for reform formulation
(1)
 PEFA report is one – out of several - inputs
– Identification of main strengths and weaknesses – and
potential impact on budgetary outcomes
– Other factors: political economy, institutional, culture,
constitution/legal, resources, capacity at entry
 Ownership means government decisions on priorities
– Government to consider all factors in deciding priorities
– Reform dialogue with donors to allow ample space
 Do not use Indicator scores simplistically
– A low score is not sufficient justification for high priority reform
– Other factors: relative importance of subject,
complexity/timeframe for improvement, interdependence with
other elements, weakest links

4
 Coverage of PFM Performance Report in the PFM Reform Cycle

Implement
PFM reforms

High level
Formulate performance
PFM reform overview
program

PFM-PR

Recommend Identify
Identify main
main
PFM reform
PFM reform PFM
PFM
measures
measures weaknesses

Investigate
underlying
causes

5
 PEFA reports for reform
formulation (2)
 Complementary analysis to PEFA required
– Detailed analysis of underlying causes needed for
formulation of detailed action plan
– Limit such analysis to priority areas
– Drill-down tools – some exist, others under development

 PEFA is preparing guidance on use of PEFA reports in

reform formulation
– Stock-taking of conceptual approaches used
– Country case studies illustrating good practice

6
Purposes of Standard Diagnostic Tools
PER CFAA CPAR Fiscal TA FRA PFM-
ROSC report PR
Review of public X
expenditure policies &
budgetary outcomes
Review of political X X
incentives
High level overview of X
PFM performance
Identification of PFM X X X X X X
strengths/weaknesses
In-depth analysis of X X X X X
capacity factors
Recommendations for X X X X X
reform
Assess risk to public X X
funds
Track progress over time * * * X
* if performance indicators are incorporated

7
 Country case - Norway
 Findings of Norad-managed self-assessment presented to
OECD-DAC in December 2007
 The assessment showed low scores for seven areas
of PFM system performance
 Ministry of Finance reaction:
– Weaknesses in procurement practices and follow-up to
external audit findings need to be addressed.
– Three areas of low scoring not considered priority at
present (Multi-year program/sector budgeting, Limited
extent of internal audit, no consolidated overview of
risks from autonomous agencies and public
corporations)
– Two indicators scored low but are municipal
responsibilities; central government will not get
involved.

8
 What can countries use the
PEFA Framework for?

 Harmonize information needs

 Inform PFM reform formulation,
priorities
 Compare to and learn from peers
 Monitor results of reform
efforts

9
 PEFA reports for country
comparison (1)
 PEFA Framework developed for in-country use
– ‘Summary assessment’ provides nuanced overview of strengths
and weaknesses as basis for reform prioritization
– No method given for arriving at one measure for ‘overall
performance level’
 Wide interest in country comparison using aggregation
– Researchers - learning on determining factors
– Donors - aid allocations
– Governments - peer learning
 Aggregation requires three decisions
– Conversion from ordinal to numerical scale
– Weighting of indicators (generally and by country)
– Weighting of countries (for country cluster analysis)

10
 PEFA reports for country
comparison (2)
 No scientifically correct or superior basis for deciding
conversion and weights exists
– Each user takes those decisions on individual opinion
– PEFA program will not endorse any particular method
 Preferred method of country comparison:
– A nuanced comparison of two assessment reports
– Consider country context, ensure comparison of like with like
 In case aggregation is desired:
– Be transparent on aggregation methods used
– Discuss reasons for choice
– Sensitivity analysis to illustrate impact on findings

11
 What can countries use the
PEFA Framework for?

 Harmonize information needs

 Inform PFM reform formulation
and priorities
 Compare to and learn from peers
 Monitor results of reform
efforts

12
Repeat Assessments

 As at March 2009, thirteen ‘real’ repeat

assessment undertaken
 Expected to become common from
2008/2009 onward i.e. three years after the
first series of baseline assessments
What we want to determine?

 Specificchanges in system
performance
– What has changed?
– How much?
Comparison of Indicator Scores
 Indicatorscores will provide a crude
overview of changes over time, but …
– Dimensions may change differently
– Performance may not always change enough
to change the score (use of arrow)

 So more detailed explanation is required

Other Possible Reasons for
Change in Scores
 Changes in definitions
 Improved availability of or access to
information
 Different information sampling &
aggregation
 Scoring methodology mistakes in previous
assessment
Reporting on Progress Made
 Explain all factors that impact a change in
rating indicator-by-indicator
 Identify the performance change
 Ensure that any reader can track the
change from the previous assessment –
what performance change led to the
change in a rating
Reporting on Progress Made
Indicator Score Score Performance change Other factors
2005 2007

PI-1 C B Performance appears Not clear if all external

improved based on project funds were
2005: last deviations excluded from data
6%, 11%, 18% for 2005 assessment
but probably
2007: 5%, 11%, 6%
insignificant issue.
PI-4 (i) A C Performance change is 2005 assessment
uncertain, despite used data on pending
reported arrears payment orders only,
increase from 1% in not overdue invoices.
2005 to 6% in 2007.
Organizational Requirements
 Performance tracking to be clearly
reflected in the TOR
 Use of same assessment team desirable
but rarely possible
 The lead agency of the previous
assessment, if different, should assist with
access to previous assessors’ notes
 … and be part of the reference group for
the repeat exercise
 Country case - Mozambique
 PEFA incorporated in PFM reform monitoring system
– baseline 2005, repeat assessment 2007, planned follow-up
2010
 Important performance improvements 2005-2007 in budget
execution
– revenue administration, cash management, internal controls
 Improvements resulted from:
– Reforms already well under way in 2005 (e.g. IFMIS/SISTAFE
and revenue administration)
– Small managerial/admin changes (including quick-wins
identified on the basis of 2005 assessment)
– New reform initiatives in pay-roll control (identified from the
2005 assessment as an important neglected area of reform)

20
Thank you for your attention