Performance Budgeting and Performance : Lessons from the PART Initiative

COMPILED BY Mr Joseph Wachira

Overview
Background PART Process PART Questions Challenges How OMB (USA) Manages the PART Initiative
2

What is the PART?

The Program Assessment Rating Tool is a diagnostic tool used to assess program performance and to drive improvements. The PART is designed to provide a consistent approach to assessing and rating programs across the Federal government. PART assessments review overall program effectiveness, from design through implementation and results. Once completed, PART reviews help inform budget decisions and identify actions to improve results. Agencies are held accountable for implementing PART followup actions, i.e., improvement plans, for each program.

When We Began
Many systems in place to collect and report data Unclear relationship between strategic and annual goals Tendency to measure what we could instead of what we should Uneven attention to performance measurement Lots of measures, but priorities not transparent Performance data used more for reporting than decision-making
4

Where We Are Today

Distribution of Cumulative Ratings 2002 - 2006
100% 80% 60% 40% 20% 0% 2002 234
Res sN

6% 24%

11%

15%

15%

17%

26% 15% 5% 20%

26%

29%

30%

26% 5% % 50% 38% 29%

28% % 24%

28% % 22%

2003 407
Demons ed

2004 607
Adequate

2005 793
Moderately Effective

2006 977
Effective

Ineffec ve

PART Process
The PART questionnaire is divided into four sections: program purpose/design, planning, management, and results/accountability. Answers must be clearly explained and cite relevant supporting evidence, such as agency performance information, independent evaluations, and financial information. Answers translate into section scores weighted to generate an overall score and rating: Effective, Moderately Effective, Adequate, Ineffective. Programs without performance measures or data are rated Results Not Demonstrated. Additional questions are asked about particular types of programs: Block/Formula Grant, Capital Assets and Service Acquisition, Competitive Grant, Credit, Direct Federal, Regulatory-based , Research & Development.

200 PART Schedule

200 PARTs identified - January . PARTWeb available for data entry - January 22. Questionnaire guidance available - January 2 . PART training for OMB and agencies - mid February. PART drafts due - March 0. Consistency check & performance measures review - May - 0. OMB revises PARTs & passes back results to agencies - May Agencies submit appeals - May 2 . Summaries & improvement plans ready - July . Data entry locked - August . PARTs published on mid August.

PART Questions
Four sections
I. Program Purpose and Design (20%) II. Strategic Planning (10%) III. Program Management (20%) IV. Results (50%)
8

Section I: Program Purpose and Design

6(pp. 6-22) 20 weight of total score

Clarity and relevance of program purpose Soundness of program design Addresses programs structural issues Clear design and purpose an essential for identifying performance measures
9

Section II: Strategic Planning

(pp. 2 0

of total score, with links to Section IV questions

Addresses program plans and approach to long-term goals Programs must have long-term and annual performance measures and ambitious targets Emphasizes independent, quality performance evaluations, plus budget transparency and budgetperformance integration

10

Section III: Program Management

(pp.
20 of total score

Addresses: Accountability of managers, performance of partners Coordination with related programs Financial management and efficiency improvements Correction of deficiencies Do programs have procedures in place to measure and achieve efficiencies and cost effectiveness?
11

Section IV: Program Results/Accountability

(pp.
0 of total score

-6 )

Assesses achievement of long-term and annual performance and efficiency goals Compares actual performance to targets (identified in Sections II and III) Effectiveness in achieving goals based on independent evaluations Compares with performance of similar programs
12

Performance Measures are Central to the PART

Allows tailoring to the specific program Results are the most valuable information product of the PART Takes most time in completing the PART Biggest determinant of overall score and rating Updated regularly to help keep PART information current
13

How PART improves performance measurement

Outcome-oriented long-term measures reflecting program purpose Outcome-oriented annual measures that directly support longterm goals If goals are outputs, must explain how they reflect progress toward desired outcomes Challenging but realistic quantifiable targets and timeframes Clear baseline from which to measure changes in performance Credit in results section tied to measures in strategic planning section Performance measures used to manage Accountability for achieving performance goals

14

Where performance measures are today

Of programs assessed 6 have long term measures

2 have ambitious targets for long-term measures have annual measures

2 have ambitious targets for annual measures

15

Examples of Improved Measures

Coast Guard Aids to Navigation Old focus: Percentage of time radio navigational systems available Current focus: Five year average of number of collisions, allisions, and groundings National Bone Marrow Donor Registry Old focus: Number of donors in registry Current focus: Number of transplants facilitated and post-transplant survival rate Leaking Underground Storage Tanks Old focus: Number of clean-ups completed Current focus: Number of clean-ups that exceed state risk-based standards for human exposure and ground water migration Community Health Centers Old focus: Numbers and characteristics of persons served and services provided Current focus: Heath outcomes such as low birth weight babies Small Business Development Centers Old focus: Number of small businesses counseled or trained
16 Current focus: Number of jobs created (new businesses v. old businesses)

Performance Measures
Outcome: Events or conditions external to the program and of direct importance to the public, beneficiaries and/or customers. They relate to the programs mission, purpose, and strategic goals. Output: Internal program activities products and services delivered to the public, beneficiaries. Efficiency: Reflect economical and effective acquisition, use, and management of resources to achieve program outcomes or produce program outputs. Outcome efficiency Output efficiency Input productivity

17

Performance Goals
Targets Improved levels of performance needed to achieve stated goals. Programs must have ambitious but realistic, achievable targets and timeframes for performance measures. Together, measures, targets, and timeframes establish the programs performance goals.
18

Program Evaluations
Scope - Examine underlying cause and effect relationship between program and achievement of performance targets. Independence - Performed by non-biased parties with no conflict of interest. Quality Applicability All programs expected to undergo some type of evaluation. Impact Prefer effectiveness evaluations (outcome, e.g., whether Federal intervention makes a difference). Rigor The most rigorous evidence that is appropriate and feasible for that program. 19

Does It Ever End?

Steps after PARTs are completed Draft summaries for ExpectMore.gov Spring Updates in PARTWeb Complete Improvement Plans
All programs must have, regardless of PART rating Focus on findings in the PART assessment Implement plans and report on progress

ExpectMore.gov release mid-August

20

Challenges: Lessons to Learn Quickly

Share drafts, communicate frequently. Use clear, direct language. Stick to deadlines. Dont take the PART personally. Rely on evidence, not anecdotes.
21

Challenges: Measurement
Uneven quality of performance measures in PARTs Several areas difficult to measure Increasing the timeliness of performance reporting
Consistency: agencies and OMB answer some questions differently
22

Challenges: Program Evaluation

Want to promote evaluation to measure and improve program design, implementation, and effectiveness, including cost-effectiveness.. Evaluations are not used enough to assess impact and improve performance
Decision makers do not appreciate and, consequently, do not routinely invest in evaluations. Technical complexity can make them hard to understand and thus undermine confidence in results.

23

Challenges: Improvement Plans

Aggressiveness varies Unclear how they impact program results (versus PART score) Uneven attention to plans across agencies and OMB

24

Challenges: Improving Performance Improving PART score versus improving performance Ensuring that program managers are empowered and accountable Assessing improvement plans fairly Sharing good approaches and models
25

Challenges: Impact
Executive Branch
Management, funding, or authorization decisions are not regularly based on the PART Presidents Management Agenda Budget and Performance Integration initiative is being used to leverage greater use of PART results

Congress
Rare, diverse references to PART Not the basis for legislative action Few oppose vigorously

Crosscutting
Opportunity for collaboration among like programs

26

27

ExpectMore.gov Summary

28

PART Resources Online

www.partburundi.blogspot.com
Information on process and schedule Guidance for completing PART PART team Supporting materials

29

Case study:How OMB(USA) Manages the PART Initiative

30

Presidents Management Agenda

A strategy for improving Federal management and performance with five government-wide and nine agency-specific goals. Strategic Management of Human Capital Competitive Sourcing Improved Financial Performance Expanded Electronic Government Budget and Performance Integration

The President directed agency heads to designate a Chief Operating Officer for day-to-day operations. The President designated the Presidents Management Council (PMC) as an integrating mechanism for policy implementation across government, headed by OMBs Deputy Director for Management and comprised of the COOs.

31

32

BUDGET AND PERFORMANCE INTEGRATION

Criteria for Achieving GREEN
Senior managers meet at least quarterly to examine reports integrating financial and performance information for all major Department responsibilities. Agency works to improve program performance and efficiency each year; Strategic plans contain a limited number of outcome-oriented goals and objectives. Annual budget and performance documents incorporate measures identified in the PART and focus on the information in the senior management report; Reports the full cost of achieving performance goals accurately in budget and performance documents and can accurately estimate the marginal cost of changing performance goals; Has at least one efficiency measure for all PARTed programs; Uses PART evaluations to direct program improvements and hold managers accountable for them, and uses PART findings and performance information to justify funding requests, management actions, and legislative proposals; and Less than 10% of agency programs receive a Results Not Demonstrated rating for two years in a row. To maintain green status, agency: Improves program performance and efficiency each year; and
33

Uses marginal cost analysis to inform resource allocations, as appropriate.