Assignment 3: Program Evaluation Plan Draft

PLATO Credit Recovery Program

Group 15
Jennie Champion, Andrea Ryles, and Wendy Willoughby

MEDT 84801: Fall Semester 2016

University of West Georgia

Background and Program Evaluation

Description of the Program Begin Evaluated

The PLATO Program is a math credit recovery program at Starrs Mill High School in
Fayetteville, Georgia that was first implemented last year. Starrs Mill High Schools mission is
to empower individuals to learn, succeed and serve (Starrs Mill High School, 2016). The
PLATO program helps Starrs Mill remain faithful to their mission statement by providing lower
performing students who have failed Algebra I or Geometry the opportunity to receive additional
instruction to not only recover credit needed to graduate, but to master the concepts necessary to
be successful in math.
The program partners with Edmentum, an online resource that provides online courses to over
8,000 districts in the United States (Edmentum, 2016). Although the Edmentum offers courses in
a wide range of subjects, Starrs Mill High School only uses PLATO for their Algebra I and
Geometry credit recovery programs. The program is aligned with the common core and is
offered to those students who did not pass Algebra I or Geometry. Classes are held for one hour
a day during the regular school day. During that time period, students are able to move at their
own pace through various modules that are filled with instructional and practice activities. Each
student is also assessed before advancing to the next lesson or module. In fact, a student cannot
advance to the next level until he/she has achieved at least 80% mastery of the content.
Although the long term goals of the program are to have students recover lost credit, increase
the number of students who graduate high school within four years and to get lower performing
students out of the failure cycle, there are other goals as well. The short term goals are to allow
students to work at their own pace and gain confidence in their math skills. Due to PLATOs rich
media and interactive content, Starrs Mill is able to achieve such short term goals. Along the

way, the program can address its intermediate outcome, which pertains to monitoring student
progress via periodic quizzes and tests which allow students to gauge their skill level. There is
also a certified teacher available to monitor and assist the students.
PLATO even monitors student attendance and the time a student spends on each module. This
helps teachers and counselors follow up with students on their progress and intervene when a
students motivation appears to be low, which again addresses the intermediate outcome of the
logic module (Figure 1). It is not only feasible to acquire such data, but such data is valid in
determining student progress and motivation, which are an inherent part of the intermediate
outcomes.
Evaluation Purpose

The evaluation will not only focus on breaking the cycle of failure of students in Algebra
I or Geometry, but will it also allow students the opportunity to receive additional instruction to
recover credit needed to graduate and master the concepts necessary to be successful in math.
The purpose of this evaluation plan is to determine if the blended learning environment offered
by PLATO meets the evaluation requirements of Starrs Mill High School, ensuring that the
students participating in this program are achieving the proposed output and outcome measures,
which includes short, intermediate, and long term goals.
The evaluation of this program will be in the form of a formative effort. Since this
program has only been implemented for one school year, a formative effort would provide
information on how the program needs to be revised and modified to best meet the needs of the
stakeholders. Interim results of the program will be shared with the administrative team of
Starrs Mill High School to demonstrate to what extent the program has broken the failure cycle
of students enrolled in Algebra I and Geometry. Final results of the evaluation will be shared
with Fayette County Board of Education to demonstrate to what extent the program has

increased the graduation of rate of students utilizing the PLATO credit recovery program.
Results will also be shared with parents and counselors to demonstrate to what extent the
program creates an encouraging learning environment for students. By giving attention to all
stakeholders invested in the PLATO recovery program, the range of needs of all stakeholders
associated with this program are being evaluated and affected by the outcome of the evaluation.
The following three key questions are addressed in the evaluation plan:

To what extent has there been an increase in the percentage of students that
graduate high school in four years since the implementation of the PLATO credit
recovery program?

To what extent are students who earn credit through PLATO credit recovery
successfully completing the following math courses, Algebra I and Geometry,
therefore breaking the cycle of failure?

To what extent do updates given to parents and counselors create an encouraging

environment for students?

The logic model constructed by the evaluation team, as shown in Figure 1, represents how these
questions align with the programs theory.

Figure 1
Using the graduation rates of Starrs Mill students supplied by the Fayette County Board
of Education on their website, the Board of Education of Fayette County will be able to
determine the effectiveness of PLATOs blended learning environment. With students
participating in modules and module practice, teachers assisting students with problems during
module practice, students taking module quizzes, and students completing four unit tests leads to
the recovery of the loss math credit that students had incurred. By receiving this recovery of the
lost math credit, students are able to graduate within four years, thus increasing the graduation
rate of students from Starrs Mill High School.
By successfully completing Algebra I or Geometry math courses through the PLATO
credit recovery program, students are breaking the cycle of failure. Students possess a greater
knowledge of Quadratics, Triangles, Right Triangle Trig, and Operations with Polynomials. This
breakage in the cycle of failure is shown with module completion and module quiz scores for a
Quadratic unit, Triangle unit, Right Triangle Trig Unit, and Operations with Polynomials Unit.
Students also show understanding of such concepts by successfully completing unit tests with
passing scores, receiving individual help from teacher when needed, and becoming more
independent learners through a self-paced program.
Surveys administered to former and current students will determine to what extent the
PLATO credit recovery program achieved an encouraging learning environment. The teacher
sends updates on progress and logs of progress to counselors and parents. By having the teacher
participate in these activities, parents and counselors monitor progress and encourage students.
The graduation rates obtained from the Fayette County Board of Education will
determine to what extent the percentage of students participating in the PLATO credit recovery

program graduate on a four-year time line, which will verify the outcome of the long term goals
of the credit recovery program. Student data recovered from the Statewide Longitudinal Data
System (SLDS) and Infinite Campus will determine to what extent the cycle of failure was
broken after receiving credit for Algebra I or Geometry after participating in the PLATO credit
recovery program, which will verify the process of the credit recovery program. Surveys
administered to past and present students and parents will be used to determine to what extent the
students felt as though they were in an encouraging learning environment while participating in
the PLATO credit recovery program, which will verify the outcome of the program.
Through the evaluation process of a program, stakeholders associated with that program
find how the evaluation process and product is imperative to identifying and fulfilling their
needs. As the evaluation of the PLATO credit recovery program occurs, negotiated purposes will
occur. Continuous discussions between the evaluation team and the stakeholders will occur
based on needs found during the evaluation process. While participating in the evaluation
process, the team will collect relevant information that is pertinent to the evaluation questions. In
order to obtain relevant information for the evaluation, the evaluation team will engage in
activities, descriptions, and judgements that support a meaningful process and product. By
securing a meaningful process and product, stakeholders will be able to reexamine their
understandings and beliefs about their program.
Evaluation Team
The evaluation team members of this program evaluation include Jennie Champion,
Andrea Ryles, and Wendy Willoughby. Ms. Champion is a first grade teacher at Western
Elementary School with the Coweta County School System. She has over fourteen years of
experience in education. Ms. Champion has served for three years as grade level chairperson

during her teaching experience. She has also served on the AdvancEd Accreditation Team for
her school during the 2014-2015 school year. Currently, she is serving as a mentor for a first year
teacher in her school.
Andrea Ryles is a Spanish teacher at Brookwood High School in Gwinnett County. She
has twenty years of experience teaching at both the elementary and high school level and is a
member of AATSP (American Association of Teachers of Spanish and Portuguese). She has not
only led school trips abroad, but she has also been head sponsor of the Brookwood High School
National Spanish Honor Society since 2015.
Wendy Willoughby is a high school math teacher at Starrs Mill High School. She has
twenty years experience teaching middle and high school math. She is a member of the Fayette
County Curriculum Revision Team representing Algebra I, Geometry, and Algebra II
courses. Ms. Willoughby is a former teacher of the PLATO online math recovery course and
currently assists other teachers in managing online courses. During the first year of the program,
Ms. Willoughby worked directly with the Mark Henderson, Fayette Countys Math Curriculum
Coordinator, Deb Troutman the Starrs Mill administrator overseeing the program, and the
Starrs Mill Guidance department on the development and facilitation of the PLATO online math
course. This team will work with Mark Henderson and Deb Troutman, who are key stakeholders
in the program. The evaluators will also work with Allen Leonard, an additional stakeholder and
the principal of Starrs Mill High School.
Although Ms. Willoughby is connected to the program, the other two evaluators, Ms.
Ryles and Ms. Champion, have no connection or vested interest in the programs outcome;
therefore, the evaluators are able to avoid bias and attain a high level of political viability. This

helps strengthen the evaluation by reducing any conflict of interest; thus creating an evaluation
that allows for balanced reporting.
Methodology
Data and Instrumentation
When addressing the following evaluation questions, the team will collect both existing
and new data. The data sources include graduation rates from 2014-2017, student records, and
surveys.

To what extent has there been an increase in the percentage of students that graduate
high school in four years since the implementation of the PLATO credit recovery
program? Data collected on this evaluation question is existing quantitative
data. The information is available from the Fayette County Board of Education
website. Years 2014-2016 are currently available, but 2017 will not be released until
Summer of 2017. The data represents graduation rates of all students at Starrs Mill
High School. The evaluation team can compare data to look for changes in the data
prior to the use of PLATO with the data available for after using PLATO for credit
recovery. This data is defensible and is also available from other sources, such as the
Georgia Department of Education.

To what extent are students who earn credit through PLATO credit recovery
successfully completing the following math courses, Algebra I and Geometry,
therefore breaking the cycle of failure? Data for this question will be obtained from
student records of those students who participated in PLATO credit recovery during
the 2015-2016 school year. Information will be gathered on the status of completion
of additional math courses taken by students after their successful completion of the

PLATO recovery program. This existing data of course grades can be acquired from
the student information systems, Infinite Campus (IC), at Starrs Mill High School
and the Statewide Longitudinal Data System (SLDS) at the end of December 2016
and again in May 2017 once grades are submitted. All information collected will be
coded with no record of student names to protect the rights of participants.

To what extent do updates given to parents and counselors create an encouraging

environment for students? The team will collect data for this question through the use
of a survey for students and parents of students who have taken or are taking a
PLATO credit recovery course for mathematics. The data collected from this survey
will be both quantitative and qualitative. This mixed method approach will include
some questions in Likert format and an open ended response. The student and parent
surveys can be found in appendix 1 and 2. The surveys will be delivered through
email in a Google form format.

The data chosen for questions one and two represent valid information that is published
or available from different reliable sources. The data from question three will be collected using
attitude scale related to the student updates to parents and counselors creating an encouraging
environment for students. The information gathered will be secured data. All identifying student
information will be eliminated from documents. All student and parent surveys will be coded to
pair for comparison purposes.
Sampling
With first two evaluation questions, data will be collected on all participants therefore no
sampling method is required. The first question on graduation rates will require data collection
on graduation rates representing the entire student population which ranges on average from 350-

425 students per year. The second question regarding breaking the failure cycle will focus on
forty students who previously participated in the PLATO credit recovery course. For the third
evaluation question on encouraging environment, a survey will be given to current and former
PLATO credit recovery students and their parents. The sampling method will be a stratified
random sample. The sample will include 60 students. Fifteen students who participated in the
PLATO recovery course in Fall of 2015, 15 from Spring of 2016, and 15 from Fall of 2016, and
15 who will participate in Spring of 2017 will be randomly selected for student surveys and will
represent students who have participated in the PLATO program. In addition to student
surveys, sixty parents of students who are chosen for student surveys will be given a parent
survey. The parent sample is a purposive sampling method needed to draw comparisons with
student responses.
Analysis
The evaluation questions for the PLATO credit recovery model are descriptive designs
because they describe status as defined by Marvin Alkin (2011). For each question, a standard
is addressed below.

To what extent has there been an increase in the percentage of students that graduate
high school in four years since the implementation of the PLATO credit recovery
program? To analyze this data, a direct comparison of percentages will be made
where the standard is an increase of 2 percent of students graduating over a four-year
period.

To what extent are students who earn credit through PLATO credit recovery
successfully completing the following math courses, Algebra I and Geometry,
therefore breaking the cycle of failure? To analyze this data, the evaluator will

compare data of all participants with a trend line showing the number of repeat
failures per semester after completion of the PLATO credit recovery program. The
standard for this question is to show an inverse relationship with the number of
students who fail additional math courses following completion of the PLATO credit
recovery program decreasing as the time the programs existence increases.

To what extent do updates given to parents and counselors create an encouraging

environment for students? To analyze this data, the evaluator will summarize the data
by groups (students and parents). There is not set standard for this question because
this question addresses an intermediate outcome.

Standards/Benchmarks for Evaluation Questions

As already addressed in the analysis section, the evaluation questions and the standards for
each are as follows:

To what extent has there been an increase in the percentage of students that graduate
high school in four years since the implementation of the PLATO credit recovery
program? The standard set for this question is a 2% increase of students graduating over
a 4-year period. Starrs Mill is already a high performing school with a 94% graduation
rate, so raising the benchmark to 96%, although only a 2% increase in a 4-year period, is
an ambitious, yet attainable goal.

To what extent are students who earn credit through PLATO credit recovery successfully
completing the following math courses, Algebra I and Geometry, therefore breaking the
cycle of failure? The standard set for this question is to show a decrease in repeat failures
in math for participants in the program. The benchmark is to have fewer students need to
do credit recovery once they have already participated in credit recovery; thus showing

an inverse relationship with the number of students who fail additional math courses
following completion of the PLATO program.

To what extent do updates given to parents and counselors create an encouraging

environment for students? Due to this being an intermediate outcome, there is no set
standard in place. However, proprietary standards as outlined in the text Program and
Evaluation Standards (Yarborough, Shulha, Hopson, & Caruthers, 2011), will be
respected as the rights and identity of the participants will be protected.

The evaluators must also determine what approach will be take in evaluating the program.
According to Fourniers article Establishing Evaluative Conclusions (1995) a causal approach is
used when determining the effectiveness of an intervention. Since PLATO is a credit recovery
program, it can be viewed as an intervention program. One can define PLATOs effectiveness as
the problem, which is one of the parameters listed in a working model. Furthermore, the
evaluators will attempt to determine whether or not the program is any more effective than not
having a program at all, which falls within the question parameter of the same working model.
The software program itself, the teacher assistance, and contacts made with parents and
counselors may be defined as the set of treatments to reach an outcome, which is a part of the
phenomenon parameter. The claim, another parameter, is the process of establishing a causal
conclusion. In this program, the evaluators will attempt to conclude to what extent PLATO
causes the number of students that graduate in four years to increase. School records of
graduation rates will be used to attempt to draw such conclusions. Another claim refers to
determining whether or not the number of repeat failures will decrease via student participation
in the PLATO program. That data may be attained via student records. The final claim is to
establish a connection between parent/student updates and an encouraging learning environment.

A sample of parents and students will take surveys which will help the evaluators to conclude
whether or not such updates create such an environment.
One may refer to the visual below to understand the need for a causal approach in this
evaluation:

References
Alkin, M. C. (2011). Evaluation Essentials: From A to Z. New York, NY: The Guilford Press.
Edmentum. (2016). Retrieved from http://www.edmentum.com
Fournier, D.M. (1995). Establishing evaluative conclusions. New Directions for Evaluation, 68.
15-32.
Geogria Deparment of Eduation (Nov 9, 2015) Retrievd from https://www.gadoe.org/ExternalAffairs-and-Policy/communications/Pages/PressReleaseDetails.aspx?PressView=
default&pid =395
Starrs Mill High School Website. (2016). Retrieved from http://www.fcboe.org/smhs
Yarbrough, D.B., Shulha, L.M., Hopson, R.K., & Caruthers, F.A. (2011). The program
evaluation standards. London, United Kingdom: Sage Publishing.

Appendices
Appendix 1: Survey instrument for Evaluation Question 3

Appendix 2: Parent Survey instrument for Evaluation Question 3