Inform

program implementation
Inform day to day management
Inform decisions

Traks inputs
Performance
Monitoring

Vs Expected Results

Traks
Activities
Using administrative data

Outputs

MAkes comparisons

Across programs

Analyzes trends over time

Occasionally include outocomes

Periodic

Of planned
A)Baseline data provide
information on benefeciaries
and groups before implementation
B)Baseline data ensure that the groups are similar.
C ) BAselines can assess targeting efectiveness, whether is
goig to achieve gols or not.

Objective Assessment

Generate valid
counterfactual

A)Defining measures of programs success

focuses the evaluation and the program
on intended results.
B)Value for informing important policy decisions.

Ongoing
C ompleted

Baseline data collected

prior implementation
for T&C groups

Design

Reasons

Answer questions

Asses impact beforeprogram

implementation

Implementation
Results

Simultanously with
the program design

Are built into

program implementation

Likely produce strong

Evaluation results

Depends on the context, previous data

and is neverguaranteed

C arried out at discrete times

C an produce debatable evidence

Seeks outside perspective*

Prospective
Retrospective

Determine what's taking place

Processes

Evaluations
Descriptive questions

C onditions
Describes
Org. Relationships

Categories

Stakeholders viewws

Subtopic

C ompares

WHAT is taking place Vs. WHAT should be taking place

Adress 3 types of questions

Start out with a pilot
(Including an IE)
The cost of an IE may be justified

Yes

Adress important and new policy innovations?

Assesses activities
No

Evidence exists to show that the program works in similar

cirmstances?

High

Normative questions

What are the stakes of this program?

Yes

Inputs

Methods that support Evidence-Based Policy

C an apply to

NO Worth evaluating

Low
Outputs

Deciding whether to Evaluate

Worth evaluating

Yes

Not worth evaluating

NO

Activities

Program or Policy
Does or will the program require a large portion of the
availabe budgets?

Examines outcomes
C ause-and-effect questions
Assess difference interventions makes in outcomes

Worth evaluating

Yes
Does or will the program affect a large # of people?

Not worth evaluating

NO
Expand a Program
Officials

To Obtain Budget Allocations

C ontinue a Program

Por qu evaluar?
C ountry Level

Ministries

C ompete to Obtain Founding from Finance

Governments & Ministries

Stakeholders

Accountability

Provide Tools
Governmet level

Interest in C onvincing constituents of positive returns.

Performance of government programs

To built transparency

Innovative

Program to be Impact Evaluated

Testing a new,promising approach

Global level

Impact Evaluation

C an be scaled
Replicable

Outside

C an be applied in a different setting

Evidence-Based Policy Making

If requirements checked

Should be

Policy Makers C ivil Society

Generate substantial savings

Strategically Relevant

What is the Cost-Benefit balance for a given program?

Effectiveness Studies

C over a large # of people

How do various program implementation alternatives

compare in cost.effectiveness?
Untested

C ost-Benefit Analysis

Estimates the total expected benefits of a program,

compared to its total expected costs. Quantify in monetary
terms.

Identify wich investments yield the high rate of return

Provide evidence
from normal cirumstances

Provide true evidence for beneficiaries and

other outside the sample (external validity)

C ost-Efectiveness

Quality

Little is known about local or global affectiveness

Influential

Demand

Impact Evaluation
(Monitoring & Evaluation)

Efficiency
Verify & Improve

Results inform key policy decisions

Effectiveness of interventios at various levels of

implementation.
To Focus on Results

C ompares the relative performance of two or more

programs or program alternatives in reaching a comon
outcome.

Allow policy makers to make informed decisions to invest

Of individuals
Asses changes in well-being
From a particular program, policy or project

Estimates average impacts of a program on welfare

Used to test alternative program options
(X vs Y)

Inform policy decisions

Tendency

Provide Evidence
Shape public opinion

Enhance Accountability

From inputs to outcomes & results

Pay for -Perfomance incentives

Used for managers to

Public Service Providers

Inform Budger Allocations

Resahpe how public

policies are carried out

Subtopic

WHAT IS TE IMPAC T OF A PROGRAM ON AN OUTC OME OF

INTEREST?
Seeks to answer cause-and-effect questions

Guide Policy Decisions

Only ininterested in the impact on outcomes

Looks for changes attributable to the program

WHAT IS?
Participants in the program
Requires comparison group
No participants in the program

Focus on causality

Combining Sources of Information

Monitoring Data

Estimates C ounterfactual

Outcome for no participants in the program

Determines the methodologies that can be used

Qualitative Data

C ontrol group
Relies on comparing

WHO received the program

Speed of the program's expansion
HOW resources are spont
WHAT activities are being implemented as planed
Provide info on the costo of implementation

Treatment group

May not be representative of the program's beneficiaries

Provide complementary
perspectives on a pgorgram's performance
from selected beneficiaries & key informants

C urtailing inefficient programs

Sacaling up interventions that work

Focus groups

Inform policy makers

Adjusting program benefits

Interviews

C ontrol Group
Select among program alternatives

For Policy Decisions

"Treatment arms"
Treatment Group

C omparing approaches
Test effectiveness of program implementation alternatives

When a program can be implemented in several ways, wich

one is the most effective?

For subsequent programs

Provide credible evidence

Is a given program effective compared to the absence of the

program?

When Designing Stage, evaluators use Qualitative data to

develop hypotheses as to HOW and WHY the program will
work.
Intermediate stage. before quantitaive results qualitative can
provide to policy makersquick
insights into what is happening in the program
Stages
Provide context and explanations for quantitative results.

Analysis Stage

To get inside the "black box" of what happened in the

program
To explore cases of success and failure

Process Evaluations

Focus on HOW a program

is implemented and operates

Documents program's
development and operation

Relatively quick and at a

reasonable cost

Valuable in the initial stages

of a program and pilots

Informs how to improve

program implementation

To Built Knowledge

Effectiveness of programs