Final Project: Program Evaluation

Final Project: Program Evaluation

Ralph Herndon
Melisa Eichele
Krystle Leanza
FRIT 7237
Dr. Lucas Jensen

Implementation Evaluation

Final Project: Program Evaluation

The entire program begins 6/2015 and ends by 2/2016. There is pre-program training, the
program implementation, and the follow up. Each step in the process will need evaluation. To
ensure the qualitative data is current each step will be evaluated as it is implemented. The
success of the PBL project is effected by Great Southern University and the participants. The
implementation evaluation will include each party in the proces.
Implementation Evaluation Questions:
1.
2.
3.
4.

What was the quality of the initial workshop?

What was the quality of the follow up and support activities?
Were the teachers able to effectively implement PBL in the classroom?
Did the PBL program affect student achievement?

The following outline is the schedule for the implementation evaluation:

Program
Start
Date

Implementation Step Evaluation Type

Eval
Due
Date

7/2

Summer Workshop

8/1

Great Southern
Teacher Survey: Support and follow up of Great
University follow up Southern Faculty. List of current unanswered
and support
questions/concerns.
Greater Southern University: Support/Eagerness
from teachers and administration.

90% by
8/8

10/1

Mid-semester
Classroom
Implementation

Teacher Survey: Knowledge PBL, Knowledge

NSSM, Ability to Implement, Integrating
technology, Challenges
Great Southern University Survey: Rate the
interaction with teachers, Eagerness/Diligent
teacher implementation

90% by
10/14

12/15

Classroom
Implementation

Teacher Survey: Knowledge PBL, Knowledge

NSSM, Ability to Implement, Integrating
technology, Challenges
Great Southern University Survey: Rate the
interaction with teachers, Eagerness/Diligent
teacher implementation

90% by
1/10

1/10

Student achievement Gather and evaluate student scores on

Teacher Survey: quality and value of information 90% by

& value of Great Southern Faculty
7/16

1/24

Final Project: Program Evaluation

scores

lessons/units.

1/10

Teacher &
Administration
Follow Up

Meetings/Interviews with teachers and

administration: Implementing the program and
the effectiveness of the program
Teacher Survey: Self evaluation of
implementation
Great Southern University: Meetings/Interviews
on implementing the program.

1/24

6/1

Compare CRCT
Scores

Compare the prior testing CRCT scores (20102011) and the current scores.

6/14

7/1

Complete Program
Evaluation

Use all of the above data to evaluate the entire

program and as a whole.

8/1

Summative Evaluation:
The two main factors that should be considered when conducting the summative
evaluation of this project are the extent to which teachers were able to develop PBL modules
that were connected to local businesses and industries, aligned with NSSM, and incorporated
appropriate uses of technology; and the extent to which teachers were able to implement and
evaluate those modules. In order to determine if the project was successful, a mixture of
qualitative and quantitative data should be utilized. The table below indicates the schedule for
the summative data.
Objective

Indicator

Data Source

Create PBL Modules

Modules draw on content
based on local businesses and processes from local
and industries
businesses

Teacher surveys and observations by

Great Southern University faculty;
surveys of local businesses and
industries; module reviews by GSU
faculty investigating link between
module content and business and
industry needs

Create PBL Modules

addressing NSSM

Teacher planning documentation

indicating which module lessons are
aligned with which NSSM; student

Module content, tasks,

and assessments are
aligned with appropriate

Final Project: Program Evaluation

NSSM

surveys; evaluator observation

during implementation of learning
modules

Integrate technology into

PBL experiences

PBL modules contain

activities that effectively
use technology

Teacher technology documentation;

evaluator observation during
implementation of learning modules

Implement and evaluate

PBL modules

Record of implementation
Record of self-evaluation

Teacher implementation
documentation and teacher
interviews following implementation
of modules

Data Collection Summary/Schedule

Data will be collected using a mixed methods approach. Instruments that will be used to collect
data include surveys, student achievement data, interviews, and questionnaires. The surveys will
contain rating scales that rate various components on a scale from one to five. Surveys will also
include open ended questions. Student scores will be drawn from summative and formative
assessments of lessons being taught in the classroom. In addition, students standardized test
scores will be reviewed and compared with their current standardized test scores (previous
CRCT and GMAS scores). Interviews will be conducted with all team members of the
implementation program. Interview questions will include multiple choice, rating scales, and
open ended questions. Lastly, participants will complete an overall evaluation of the program
questionnaire emailed to them to complete independently. Below is the schedule at which the
data is to be collected and analyzed.
Evaluation Plan Form
Project Director: Carlos Hedges, Ph.D.
Title of Project Proposal: New State Standards for Mathematics and Problem-based Learning

Final Project: Program Evaluation

Data set

Date of collection

Instruments
already
developed?

Data collected by:

Survey Results

7/2

Yes

Evaluation Team

Survey Results

8/1

Yes

Evaluation Team

Survey Results

10/1

Yes

Evaluation Team

Survey Results

12/15

Yes

Evaluation Team

Student Scores
on
implementation
lessons

1/10

No

Evaluation Team

Interviews and
Survey Results

1/10

No

Evaluation Team

Student CRCT
Scores and
Current GMAS
Scores

6/1

No

Evaluation Team

Overall Program 7/1

Evaluation
Questionnaire
Results

No

Evaluation Team