Professional Documents
Culture Documents
Beth Holz
EDF 5461 Introduction to Program Evaluation
By PresenterMedia.com
Agenda
History
Applications
Strengths, Limitations & Use
Summary
Sources
Item 1
Case Study
Item 2
Item 3
Item 4
Item 5
Item 6
History
One of the oldest evaluation approaches
Comprehensive judgments of the quality of a program or product
Judges the quality of products for potential customers
Used by government agencies and independent consumer advocates
Became important in educational evaluations in
mid 1960s
Not concerned with field expertise, but evaluators
expertise
Concerned with quality as MEASURED not
experienced
Based on Evaluation logic and quantitative
methods
Determine Criteria
Examine or measure performance
Synthesize information
Michael Scrivens Model
Determine criteria
Evidence of achievement of important
educational objectives
Evidence of achievement of important
non- educational objectives
Follow-up results
Secondary and unintended effects
Range of utility
Moral considerations
Costs
Scrivens Evaluation Logic - Step 1
Examine or measure performance using
Key Evaluation Checklist
Generic (products, personnel, programs, evaluation
Critical emphasis is merit and worth
Broad use, both consumers and evaluators
Fosters comprehensiveness in evaluations
Scrivens Evaluation - Step 2
Product Evaluation Checklist
Products
Critical emphasis is demission's of product soundness
Broad use, consumer and evaluators
Fosters comprehensiveness in product evaluations
Step 2 (cont.)
Step 3
Synthesize results
Provide results to consumer
The goal of the WWC
is to be a resource for
informed education
decision making.
Identifies studies that
provide credible and
reliable evidence of
the effectiveness of a
given practice,
program, or policy.
Expert, independent,
nonprofit organization
whose mission is to
work for a fair, just,
and safe marketplace
for all consumers and
to empower
consumers to protect
themselves.
Watch and study
customers reactions
Use consumer-oriented
evaluation for formative
purposes.
Applications
Public and Private Organizations and Industries
Consumers make informed choices
based on a comprehensive evaluation
of products or programs
Consumers can compare outcomes
between products or programs
Saves consumers time
Consumers are more aware of market
gimmicks and tricks
Strengths
Time required to complete such
comprehensive evaluations is passed on
to consumer in the form of higher
prices
Dependence on outside agencies to
perform evaluations
Stringent criteria might inhibit product
creation
Limitations
Summative evaluation
Consumer focused
Focused on assisting
consumers make purchasing
decisions
Formative evaluations
Organization & Consumer Focused
Evaluations occur throughout the
life-cycle of product development
Recognizes multiple stakeholders
Consumers
Organizations
Others involved in process
Use
Assists the PUBLIC understand the Quality, Value, Merit or Worth of
Product, Program, or Policy
Relies on transparent and quantitative methodologies made by an
evaluator, NOT an expert
Uses Scrivens Logic of evaluation
Although still used in government and industry, the consumer-oriented
approach is losing prominence in professional evaluator literature
Conclusion
Sources:
http://www.consumerreports.org/cro/about-us/index.htm
http://ies.ed.gov/ncee/wwc/aboutus.aspx
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2012).Program evaluation alternative
approaches and practical guidelines. (4th ed.). Upper Saddle River, NJ: Pearson
Education, Inc.
Picture from PresenterMedia.com