You are on page 1of 26

February 11–14, 2019

Results for: Johnsontown Road Elementary School


Diagnostic Review Report

Table of Contents

Introduction ............................................................................................................................... 3
AdvancED Standards Diagnostic Results .................................................................................... 4
Leadership Capacity Domain............................................................................................................... 4
Learning Capacity Domain .................................................................................................................. 5
Resource Capacity Domain ................................................................................................................. 6
Effective Learning Environments Observation Tool® (eleot®) Results ....................................... 7
eleot Narrative.................................................................................................................................. 11
Findings .................................................................................................................................... 13
Improvement Priorities ..................................................................................................................... 13
Insights from the Review .................................................................................................................. 18
Next Steps......................................................................................................................................... 19
Team Roster ............................................................................................................................. 20
Addenda................................................................................................................................... 22
Student Performance Data ............................................................................................................... 22
Schedule ........................................................................................................................................... 25

© Advance Education, Inc. 2 www.advanc-ed.org


Diagnostic Review Report

Introduction
The AdvancED Diagnostic Review is carried out by a team of highly qualified evaluators who examine the
institution’s adherence and commitment to the research aligned to AdvancED Standards. The Diagnostic Review
Process is designed to energize and equip the leadership and stakeholders of an institution to achieve higher levels
of performance and address those areas that may be hindering efforts to reach desired performance levels. The
Diagnostic Review is a rigorous process that includes the in-depth examination of evidence and relevant
performance data, interviews with stakeholders, and observations of instruction, learning, and operations.

Standards help delineate what matters. They provide a common language through which an education community
can engage in conversations about educational improvement, institution effectiveness, and achievement. They
serve as a foundation for planning and implementing improvement strategies and activities and for measuring
success. AdvancED Standards were developed by a committee composed of educators from the fields of practice,
research, and policy. These talented leaders applied professional wisdom, deep knowledge of effective practice,
and the best available research to craft a set of robust standards that define institutional quality and guide
continuous improvement.

The Diagnostic Review Team used the AdvancED Standards and related criteria to guide its evaluation, looking not
only for adherence to standards, but also for how the institution functioned as a whole and embodied the
practices and characteristics of quality. Using the evidence they gathered, the Diagnostic Review Team arrived at a
set of findings contained in this report.

As a part of the Diagnostic Review, stakeholders were interviewed by members of the Diagnostic Review Team
about their perspectives on topics relevant to the institution's learning environment and organizational
effectiveness. The feedback gained through the stakeholder interviews was considered with other evidence and
data to support the findings of the Diagnostic Review. The following table lists the numbers of interviewed
representatives of various stakeholder groups.

Stakeholder Groups Number


District-level Administrators 2
Building-level Administrators 2
Professional Support Staff (e.g., Counselor, Media Specialist, Technology 8
Coordinator)
Certified Staff 26
Non-certified Staff 17
Students 46
Parents 6
Total 107

© Advance Education, Inc. 3 www.advanc-ed.org


Diagnostic Review Report

AdvancED Standards Diagnostic Results


The AdvancED Performance Standards Diagnostic was used by the Diagnostic Review Team to evaluate the
institution’s effectiveness based on the AdvancED’s Performance Standards identified as essential for realizing
growth and sustainable improvement in underperforming schools. The diagnostic consists of three components
built around each of the three Domains: Leadership Capacity, Learning Capacity, and Resource Capacity. Point
values are established within the diagnostic, and a percentage of the points earned by the institution for each
Standard is calculated from the point values for each Standard. Results are reported within four categories: Needs
Improvement, Emerging, Meets Expectations, and Exceeds Expectations. The results for the three Domains are
presented in the tables that follow.

Leadership Capacity Domain


The capacity of leadership to ensure an institution’s progress toward its stated objectives is an essential element of
organizational effectiveness. An institution’s leadership capacity includes the fidelity and commitment to its
purpose and direction, the effectiveness of governance and leadership to enable the institution to realize its stated
objectives, the ability to engage and involve stakeholders in meaningful and productive ways, and the capacity to
implement strategies that improve learner and educator performance.

Leadership Capacity Standards Rating

1.1 The institution commits to a purpose statement that defines beliefs about teaching Needs
and learning, including the expectations for learners. Improvement
1.3 The institution engages in a continuous improvement process that produces Needs
evidence, including measurable results of improving student learning and Improvement
professional practice.
1.6 Leaders implement staff supervision and evaluation processes to improve Needs
professional practice and organizational effectiveness. Improvement
1.7 Leaders implement operational process and procedures to ensure organizational Needs
effectiveness in support of teaching and learning. Improvement
1.8 Leaders engage stakeholders to support the achievement of the institution’s Needs
purpose and direction. Improvement
1.9 The institution provides experiences that cultivate and improve leadership Needs
effectiveness. Improvement
1.10 Leaders collect and analyze a range of feedback data from multiple stakeholder Needs
groups to inform decision-making that results in improvement. Improvement

© Advance Education, Inc. 4 www.advanc-ed.org


Diagnostic Review Report

Learning Capacity Domain


The impact of teaching and learning on student achievement and success is the primary expectation of every
institution. An effective learning culture is characterized by positive and productive teacher/learner relationships;
high expectations and standards; a challenging and engaging curriculum; quality instruction and comprehensive
support that enable all learners to be successful; and assessment practices (formative and summative) that
monitor and measure learner progress and achievement. Moreover, a quality institution evaluates the impact of its
learning culture, including all programs and support services, and adjusts accordingly.

Learning Capacity Standards Rating

2.1 Learners have equitable opportunities to develop skills and achieve the content Needs
and learning priorities established by the institution. Improvement
2.2 The learning culture promotes creativity, innovation and collaborative problem- Needs
solving. Improvement
2.5 Educators implement a curriculum that is based on high expectations and prepares Needs
learners for their next levels. Improvement
2.7 Instruction is monitored and adjusted to meet individual learners’ needs and the Needs
institution’s learning expectations. Improvement
2.9 The institution implements, evaluates, and monitors processes to identify and Needs
address the specialized social, emotional, developmental, and academic needs of Improvement
students.
2.10 Learning progress is reliably assessed and consistently and clearly communicated. Needs
Improvement
2.11 Educators gather, analyze, and use formative and summative data that lead to Needs
demonstrable improvement of student learning. Improvement
2.12 The institution implements a process to continuously assess its programs and Needs
organizational conditions to improve student learning. Improvement

© Advance Education, Inc. 5 www.advanc-ed.org


Diagnostic Review Report

Resource Capacity Domain


The use and distribution of resources support the stated mission of the institution. Institutions ensure that
resources are distributed and utilized equitably so that the needs of all learners are adequately and effectively
addressed. The utilization of resources includes support for professional learning for all staff. The institution
examines the allocation and use of resources to ensure appropriate levels of funding, sustainability, organizational
effectiveness, and increased student learning.

Resource Capacity Standards Rating

3.1 The institution plans and delivers professional learning to improve the learning Needs
environment, learner achievement, and the institution’s effectiveness. Improvement
3.2 The institution’s professional learning structure and expectations promote Needs
collaboration and collegiality to improve learner performance and organizational Improvement
effectiveness.
3.4 The institution attracts and retains qualified personnel who support the institution’s Needs
purpose and direction. Improvement
3.7 The institution demonstrates strategic resource management that includes long- Emerging
range planning and use of resources in support of the institution’s purpose and
direction.
3.8 The institution allocates human, material, and fiscal resources in alignment with the Emerging
institution’s identified needs and priorities to improve student performance and
organizational effectiveness.

© Advance Education, Inc. 6 www.advanc-ed.org


Diagnostic Review Report

Effective Learning Environments Observation Tool® (eleot®)


Results
The eProve™ Effective Learning Environments Observation Tool (eleot) is a learner-centric classroom observation
tool that comprises 28 items organized in seven environments aligned with the AdvancED Standards. The tool
provides useful, relevant, structured, and quantifiable data on the extent to which students are engaged in
activities and demonstrate knowledge, attitudes, and dispositions that are conducive to effective learning.
Classroom observations are conducted for a minimum of 20 minutes.

Every member of the Diagnostic Review Team was eleot certified and passed a certification exam that established
inter-rater reliability. Team members conducted 14 observations during the Diagnostic Review process, including
all core content learning environments. The following charts provide aggregate data across multiple observations
for each of the seven learning environments.

Diagnostic Review eleot Ratings


A. Equitable Learning B. High Expectations C. Supportive Learning D. Active Learning
E. Progress Monitoring F. Well-Managed Learning G. Digital Learning

3.1
2.8
2.5
2.3
2.2
2.0

1.3

Environment Averages

© Advance Education, Inc. 7 www.advanc-ed.org


Diagnostic Review Report

A. Equitable Learning Environment

Somewhat
Observed
Indicators Average Description

Evident

Evident

Evident
Very
Not
Learners engage in differentiated learning opportunities and/or
A1 1.9 43% 29% 21% 7%
activities that meet their needs.

Learners have equal access to classroom discussions, activities,


A2 3.2 0% 14% 50% 36%
resources, technology, and support.

A3 3.3 Learners are treated in a fair, clear, and consistent manner. 0% 7% 57% 36%

Learners demonstrate and/or have opportunities to develop


empathy/respect/appreciation for differences in abilities,
A4 1.7 57% 21% 14% 7%
aptitudes, backgrounds, cultures, and/or other human
characteristics, conditions and dispositions.
Overall rating on a 4 point
scale:
2.5

B. High Expectations Learning Environment

Somewhat
Observed
Indicators Average Description

Evident

Evident

Evident
Very
Not

Learners strive to meet or are able to articulate the high


B1 1.9 43% 29% 29% 0%
expectations established by themselves and/or the teacher.

Learners engage in activities and learning that are challenging but


B2 2.3 21% 36% 36% 7%
attainable.

Learners demonstrate and/or are able to describe high quality


B3 1.7 50% 29% 21% 0%
work.

Learners engage in rigorous coursework, discussions, and/or tasks


B4 2.1 that require the use of higher order thinking (e.g., analyzing, 29% 36% 36% 0%
applying, evaluating, synthesizing).

Learners take responsibility for and are self-directed in their


B5 2.1 29% 29% 43% 0%
learning.

Overall rating on a 4 point


scale:
2.0

© Advance Education, Inc. 8 www.advanc-ed.org


Diagnostic Review Report

C. Supportive Learning Environment

Somewhat
Observed
Indicators Average Description

Evident

Evident

Evident
Very
Not
Learners demonstrate a sense of community that is positive,
C1 2.6 14% 29% 43% 14%
cohesive, engaged, and purposeful.

C2 2.5 Learners take risks in learning (without fear of negative feedback). 14% 29% 50% 7%

Learners are supported by the teacher, their peers, and/or other


C3 3.0 0% 29% 43% 29%
resources to understand content and accomplish tasks.

Learners demonstrate a congenial and supportive relationship


C4 3.2 7% 7% 43% 43%
with their teacher.

Overall rating on a 4 point


scale:
2.8

D. Active Learning Environment

Somewhat
Indicators Average Description Observed

Evident

Evident

Evident
Very
Not

Learners' discussions/dialogues/exchanges with each other and


D1 2.6 14% 36% 29% 21%
teacher predominate.

D2 1.7 Learners make connections from content to real-life experiences. 29% 71% 0% 0%

D3 2.9 Learners are actively engaged in the learning activities. 0% 21% 64% 14%

Learners collaborate with their peers to accomplish/complete


D4 2.1 29% 36% 29% 7%
projects, activities, tasks and/or assignments.

Overall rating on a 4 point


scale:
2.3

© Advance Education, Inc. 9 www.advanc-ed.org


Diagnostic Review Report

E. Progress Monitoring Learning Environment

Somewhat
Observed
Indicators Average Description

Evident

Evident

Evident
Very
Not
Learners monitor their own progress or have mechanisms whereby
E1 1.9 36% 36% 29% 0%
their learning progress is monitored.

Learners receive/respond to feedback (from teachers/peers/other


E2 2.6 7% 43% 36% 14%
resources) to improve understanding and/or revise work.

Learners demonstrate and/or verbalize understanding of the


E3 2.6 7% 29% 57% 7%
lesson/content.

Learners understand and/or are able to explain how their work is


E4 1.6 43% 57% 0% 0%
assessed.

Overall rating on a 4 point


scale:
2.2

F. Well-Managed Learning Environment

Somewhat
Observed
Indicators Average Description

Evident

Evident

Evident
Very
Not

Learners speak and interact respectfully with teacher(s) and each


F1 3.4 0% 7% 50% 43%
other.

Learners demonstrate knowledge of and/or follow classroom rules


F2 3.2 0% 14% 50% 36%
and behavioral expectations and work well with others.

Learners transition smoothly and efficiently from one activity to


F3 2.9 0% 29% 50% 21%
another.

Learners use class time purposefully with minimal wasted time or


F4 2.9 0% 36% 43% 21%
disruptions.

Overall rating on a 4 point


scale:
3.1

© Advance Education, Inc. 10 www.advanc-ed.org


Diagnostic Review Report

G. Digital Learning Environment

Somewhat
Observed
Indicators Average Description

Evident

Evident

Evident
Very
Not
Learners use digital tools/technology to gather, evaluate, and/or
G1 1.6 64% 7% 29% 0%
use information for learning.

Learners use digital tools/technology to conduct research, solve


G2 1.1 93% 7% 0% 0%
problems, and/or create original works for learning.

Learners use digital tools/technology to communicate and work


G3 1.1 93% 7% 0% 0%
collaboratively for learning.

Overall rating on a 4 point


scale:
1.3

eleot Narrative
The Diagnostic Review Team for Johnsontown Road Elementary School conducted 14 core content classroom
observations that provided team members with insight into instructional practices and student learning across the
seven learning environments. Rating averages on a four-point scale ranged from 3.1 in the Well-Managed Learning
Environment to 1.3 in the Digital Learning Environment. Although data showed that teachers were successful in
engaging students in the classroom and managing student behavior, the data also suggested a need for careful
monitoring of high expectations and rigor in instructional practices.

The Diagnostic Review Team identified seven indicators in the learning environments as strengths. Two indicators
emerged as the strongest. It was evident/very evident in 93 percent of the classrooms that students “are treated in
a fair, clear and consistent manner” (A3) and “speak and interact respectfully with teacher(s) and each other” (F1).

Other strengths emerged in the Equitable, Supportive, Active, and Well-Managed Learning Environments. In 86
percent of classrooms, it was evident/very evident students had “equal access to classroom discussions, activities,
resources, technology, and support” (A2). Instances in which students demonstrated “a congenial and supportive
relationship with their teacher” (C4) were evident/very evident in 86 percent of classrooms. Students who were
actively “engaged in learning activities” were evident/very evident in 78 percent of classrooms (D3), and in 86
percent of classrooms, it was evident/very evident that students “demonstrate knowledge of and/or follow
classroom rules and behavioral expectations and work well with others” (F2).

While student engagement in learning activities is critical, students learn best when they understand the relevance
of what they have learned. Classroom observation data showed it was evident/very evident that students made
“connections from content to real-life experiences” (D2) in zero percent of classrooms.

The Diagnostic Review Team identified the High Expectations Learning Environment as an area to leverage to
increase student achievement. It was evident/very evident in 21 percent of classrooms that students
demonstrated or were able to describe “high-quality work” (B3). Instances in which students strove to meet or
were able “to articulate the high expectations established by themselves and/or the teacher” (B1) were
evident/very evident in 29 percent of classrooms. Interview data gathered from all stakeholder groups showed
that low expectations were a part of the school culture. Students who engaged in “rigorous coursework,

© Advance Education, Inc. 11 www.advanc-ed.org


Diagnostic Review Report

discussions, and/or tasks that require the use of higher order thinking (e.g., analyzing, applying, evaluating,
synthesizing)” were evident/very evident in 36 percent of classrooms (B4).

Another area of concern for the Diagnostic Review Team was the absence of differentiated learning opportunities.
Grouping students was often observed, but students who engaged “in differentiated learning opportunities and/or
activities” that met their needs (A1) were evident/very evident in 28 percent of classrooms.

Other items to leverage for increased student learning emerged in the Progress Monitoring and Feedback Learning
Environment. Instances in which students understood and/or were able to “explain how their work is assessed”
(E4) were evident/very evident in zero percent of classrooms.

Finally, the Diagnostic Review Team identified the Digital Learning Environment as the lowest rated. The team
often observed technology being used for programs such as Dreambox or Lexia; however, the school provided no
data that reflected student progress in these programs or showed how these programs were being monitored.
Instances in which students used technology to “conduct research, solve problems, and/or create original work for
learning” (G2) and “communicate and/or work collaboratively for learning” (G3) were evident/very evident in zero
percent of classrooms. The use of technology to “gather, evaluate, and/or use information for learning” was
evident/very evident in 29 percent of classrooms (G1).

© Advance Education, Inc. 12 www.advanc-ed.org


Diagnostic Review Report

Findings
Improvement Priorities
Improvement priorities are developed to enhance the capacity of the institution to reach a higher level of
performance and reflect the areas identified by the Diagnostic Review Team to have the greatest impact on
improving student performance and organizational effectiveness.

Improvement Priority #1
Develop and monitor a systematic process to review and revise the existing curriculum in all subject areas to
ensure rigor and alignment with the Kentucky Academic Standards. Create and use a process to collect and analyze
data to monitor the quality and effectiveness of curriculum implementation and to inform instructional decisions.
Establish high expectations for all learners. (Primary Standard 2.5, Secondary Standard 1.8)

Evidence:

Student Performance Data:


A review of student performance data, as detailed in an addendum to this report, suggested that a curriculum of
high expectations and rigor with correlating instructional practices was not fully developed. Kentucky Performance
Rating for Educational Progress (K-PREP) results for Johnsontown Road Elementary School revealed that the
percentages of students who scored Proficient/Distinguished were significantly below the state averages in 2016-
2017 and 2017-2018.

From 2016-2017 to 2017-2018, third-grade and fourth-grade students had slight increases in the percent scoring
Proficient/Distinguished in reading, while fifth-grade students showed a slight decrease. In math, third- and fourth-
grade scores decreased for Proficient/Distinguished, while fifth-grade students showed an increase. During the
principal’s presentation, the principal indicated this kind of variation in scores had been common over her 18-year
tenure as principal. She stated, “One year we would meet our Annual Measurable Objective (AMO), and the next
year we wouldn’t.” The lowest percentage of students who scored Proficient/Distinguished on K-PREP was in
writing for both 2016-2017 and 2017-2018, at 14.6 percent and 16.7 percent, respectively. The student growth
indicator for 2017-2018 lagged behind the state indicator by 1.6 points. However, the 15.1 index for math was
higher than the state index by six-tenths of a point.

In the disaggregated data, the white subgroup consistently outscored other ethnic groups, with the exception of
the Hispanic students, in reading and math. Fifty percent of Hispanic students scored Proficient/Distinguished in
reading, compared to 37.8 percent of white students. In math, 37.5 percent of Hispanic students scored
Proficient/Distinguished compared to 36.9 percent of white students. Of concern to the Diagnostic Review Team
were the African-American students and students with disability with IEPs. Sixteen percent of students with
disabilities scored Proficient/Distinguished in reading; four percent in math. The African-American students also
performed at a lower level in all subject areas than their peers, except students with disabilities.

The analysis of Measures of Academic Progress (MAP) 2018 winter data revealed that percentile ranks of students
in grades three through five ranged from the third percentile to the sixteenth percentile in reading and math. The
highest Conditional Growth Percentile (CGP) was the ninth percentile in fifth-grade reading; the second highest
was the third percentile in fourth-grade math. All other SCG percentiles in reading and math were first percentiles.
The interview data revealed that some groups of students could relate their MAP scores with their targets, not

© Advance Education, Inc. 13 www.advanc-ed.org


Diagnostic Review Report

only for themselves, but for other classmates. However, for daily learning targets and activities, students were
unaware of how their work was assessed.

Classroom Observation Data:


Classroom observation data related to the High Expectations Learning Environment, as previously discussed,
revealed that the school had yet to implement a rigorous curriculum based on high expectations. Instances of
students who were able to “articulate the high expectations established by themselves and/or their teacher” (B1)
were evident/very evident in 29 percent of classrooms. It was evident/very evident in 21 percent of classrooms
that students were able to “describe high quality work” (B3). Engagement in “rigorous coursework, discussions,
and/or tasks that require the use of higher order thinking (e.g., analyzing, applying, evaluating, synthesizing)” (B4)
was evident/very evident in 36 percent of classrooms. Students engaged in “challenging but attainable” learning
activities (B2) and who were “self-directed in their learning” (B5) were evident/very evident in 43 percent of
classrooms.

Another area of concern to the Diagnostic Review Team was in the Progress Monitoring and Feedback Learning
Environment. For example, students rarely monitored their own progress toward the learning target of the day or
had opportunities to understand the metrics by which their work would be evaluated. Students had few exemplars
detailing expectations for proficiency. It was evident/very evident in 29 percent of classrooms, for instance, that
students “monitor their own progress or have mechanisms whereby their learning progress is monitored” (E1).
Finally, it was evident/very evident in zero percent of classrooms that students “understand and/or are able to
explain how their work is assessed” (E4).

Stakeholder Interview Data:


Across multiple stakeholder groups, the interview data showed that curriculum and standards work was in the
beginning stages. The Diagnostic Review Team observed that professional learning communities (PLC) met and had
some true collaboration. However, multiple stakeholder groups reported that PLC structures were inconsistent.
Interview data also indicated that although learning expectations were a regular part of the instructional process
at Johnsontown Road Elementary, these expectations failed to reach the rigor of the intended grade-level
standards. Stakeholders reported that instructional rigor was low and that teachers had low expectations for
student achievement. Parent stakeholders stated that the school had a “close is good enough” mentality. Student
work was described as “too easy” by most stakeholder groups, and students described completing worksheets
with little guidance and with self-directed use of computer-assisted programs. While walkthrough classroom
observations (PowerWalks) occurred regularly, staff members reported little feedback and virtually no coaching,
stating, “We are handed the data, and that is it.” The team found an absence of instructional coaching and
feedback from observations to improve standards-based instruction and increase the use of high-yield
instructional strategies.

District leadership and some stakeholders described school leadership as “responsive and transparent.” They
further indicated that leadership used and responded to data. The survey data largely supported those interview
statements; however, there was indication that some stakeholders feared repercussions regarding their survey
answers or had not clearly understood the questions. Some interview data described the school culture as full of
mistrust and further indicated a prevailing negative perception of school leadership. Stakeholder groups, with the
exception of a few individuals, revealed that they did not feel their voices were heard by the principal. In the 2018
School Quality Factors (SQF) Section A, the school indicated that learners did not have systemic input. Parent
support and participation were reported as low in the SQF, a viewpoint substantiated by interview data.

© Advance Education, Inc. 14 www.advanc-ed.org


Diagnostic Review Report

Stakeholder Perception/Experience Data:


In direct opposition to the stakeholder interview data and student performance data, stakeholder surveys
indicated that all three stakeholder groups believed that the curriculum was based on high expectations and
prepared learners for their next level. Eighty-six percent of staff members agreed/strongly agreed with the
statement, “All teachers in our school use a process to inform students of their learning expectations and
standards of performance” (E5). Eighty percent of parents agreed/strongly agreed that “Our school provides
opportunities for stakeholders to be involved” (D6), and 70 percent of staff members agreed/strongly agreed that
“Our school's leaders provide opportunities for stakeholders to be involved in the school” (D10).

Documents and Artifacts:


The Diagnostic Review Team reviewed many documents and artifacts submitted by the school. Among them were
the Comprehensive School Improvement Plan (CSIP), Racial Equity/Diversity Plan, Multi-Tiered Systems of Support
Plan (MTSS), 2018 Johnsontown Road School Quality Factors (SQF), and Continuous Improvement Diagnostics.
Development of these plans did not result in significantly improving student learning, as indicated by 2017-2018 K-
PREP assessment results.

The 2018 SQF contradicted classroom observation data and interview data. In the SQF, it was noted that most
learners “experience rigorous and challenging” work. However, classroom observation data indicated rigorous
instruction and higher order thinking activities were evident/very evident in 36 percent of classrooms. All
stakeholder groups remarked that rigor was lacking in the curriculum (B4). A concern that emerged in the CSIP was
that “Learners do not frequently demonstrate success on challenging goals.”

© Advance Education, Inc. 15 www.advanc-ed.org


Diagnostic Review Report

Improvement Priority #2
Develop, implement, and monitor processes that ensure instructional practices are adjusted for rigor and quality
to meet the academic needs of all students. Execute processes that monitor the quality and fidelity of instructional
practices as they pertain to differentiated learning. Collect, analyze, and use data to identify needed changes in
instructional practices and adjust practices to meet student needs. (Standard 2.7)

Evidence:

Student Performance Data:


As previously discussed in this report, the Diagnostic Review Team saw little improvement in student performance
for most groups of students. According to the reading data board located at the school, 57 percent of students in
grades three through five were below the fiftieth percentile on winter MAP scores. While no corresponding data
board existed for math, third- and fourth-grade students experienced a decline in K-PREP math scores from 2016-
2017 to 2017-2018. Disaggregated data by subgroup illustrated that the disparity in student achievement was
most pronounced in the African-American subgroup and the Disability with IEP subgroup. The student
performance data were considered, along with other data, to develop Improvement Priority #2.

Classroom Observation Data:


The classroom observation data, as previously discussed, showed that instances in which students “engage in
differentiated learning opportunities and/or activities that meet their needs” (A1) were evident/very evident in 28
percent of classrooms. It was evident/very evident in 36 percent of classrooms that “Learners engage in rigorous
coursework, discussions, and/or tasks that require the use of higher order thinking (e.g., analyzing, applying,
evaluating, synthesizing)” (B4). Small groups were observed often by the Diagnostic Review Team; however, each
group completed the same activity or worksheet with no differentiated tasks. In addition, instances in which
“Learners make connections from content to real-life experiences” (D2) were evident/very evident in zero percent
of classrooms.

Stakeholder Interview Data:


The stakeholder interview data revealed some frustration with the lack of responsiveness of some administrators
regarding the referral process for students with disabilities. Numerous stakeholders indicated that the process was
stalled or delayed significantly. Interview data indicated a need for both rigor and differentiation.

The interview data further indicated that most staff members had received training in trauma-informed care
strategies for their students, which had suggested a need for intensive, readily accessible counseling services for
students. Interview data showed that the school counselor went into classrooms once a month, not twice a month,
as indicated in the SQF.

Stakeholder Perception/Experience Data:


The stakeholder survey data was contrary to stakeholder interview data, as the survey results indicated that
stakeholder groups reported instruction was, for the most part, personalized for students. Eighty-two percent of
staff members agreed/strongly agreed that “All teachers personalize instructional strategies” (E2), and 87 percent
of parents agreed/strongly agreed that “My child’s teacher meets learning needs by individualizing instruction”
(E4).

© Advance Education, Inc. 16 www.advanc-ed.org


Diagnostic Review Report

Documents and Artifacts:


Data provided by the school for students with disabilities revealed a reduction in the number of students being
served. However, total school enrollment had declined from 425 in 2016-2017 to approximately 320 in 2018-2019.
The Diagnostic Review Team had some concerns regarding the hour-long intervention period for all students at the
end of the school day. The MTSS plan revealed a potential misunderstanding of progress monitoring, as the plan
stated, “Students are progress-monitored as needed on the partially mastered standard.” One stakeholder
commented that everyone is “in the same boat.” In the “2018-2019 Closing the Achievement Gap Diagnostic,” the
school acknowledged, “Gaps continue to persist in our overall GAP population, mainly with African Americans.”

© Advance Education, Inc. 17 www.advanc-ed.org


Diagnostic Review Report

Insights from the Review


The Diagnostic Review Team engaged in professional discussions and deliberations about the processes, programs,
and practices within the institution to arrive at the findings of the team. These findings are organized around
themes guided by the evidence, examples of programs, and practices and provide direction for the institution’s
continuous improvement efforts. The insights from the Review narrative should provide contextualized
information from the team deliberations and provide information about the team’s analysis of the practices,
processes, and programs of the institution within the five Levels of Impact: Engagement, Implementation, Results,
Sustainability, and Embeddedness.

Engagement is the level of involvement and frequency with which stakeholders are engaged in the desired
practices, processes, or programs within the institution. Implementation is the degree to which the desired
practices, processes, or programs are monitored and adjusted for quality and fidelity of implementation. Results
represent the collection, analysis, and use of data and evidence to demonstrate attaining the desired result(s).
Sustainability is results achieved consistently to demonstrate growth and improvement over time (minimum of
three years). Embeddedness is the degree to which the desired practices, processes, or programs are deeply
ingrained in the culture and operation of the institution.

Strengths:

Teachers and support staff at Johnsontown Road Elementary School demonstrated a deep commitment to their
students and one another. The Diagnostic Review Team observed many instances that demonstrated staff were
engaged with and genuinely cared about their students. The school environment was very welcoming; several staff
members greeted students each morning and intervened immediately if a student came in upset, angry, or
unhappy. Discipline was identified as an issue of concern in interview data across all stakeholder groups, but
observation data indicated leadership and staff members had implemented strategies to ensure a safe and orderly
learning environment for all students. Students were treated in a fair, clear, and consistent manner. In classroom
observation data and observations of common spaces, students exhibited knowledge of and/or followed
classroom rules and behavioral expectations. Teachers were able to understand their students’ needs through
professional learning topics such as trauma-informed care and implementation of a school wide student behavior
management system, Academic and Behavioral Response to Intervention (ABRI).

Multiple resources existed for core classrooms. Support staff members assisted to provide direct instruction to
students, and each classroom contained a class library. Staff survey data showed 89 percent of staff members
agreed/strongly agreed with the statemen, “Our school provides sufficient material resources to meet student
needs” (F3). Additionally, 86 percent of staff members agreed/strongly agreed that “Our school provides a variety
of information resources to support student learning” (F5). Technology was available for every classroom, and
students and staff were using technology for learning. The Goal Clarity Coach had created a BlogSpot for
instructional materials and data that could be accessed and edited by any teacher as needed.

The related arts program provided students ample opportunities beyond the core curriculum in art, music, physical
education, and library. The related arts teachers provided intervention during the dedicated intervention time in
the master schedule.

© Advance Education, Inc. 18 www.advanc-ed.org


Diagnostic Review Report

Continuous Improvement Process:


The interview and survey data, as well as a review of documents and artifacts, indicated that teachers and leaders
inconsistently engaged in a systematic and continuous improvement process to build instructional and
organizational capacity. The Diagnostic Review Team found no ongoing or effective use of data for decision-
making by teachers and leaders and a lack of an established schoolwide curriculum based on rigorous standards
and instructional practices. In addition, the team found no established, ongoing processes or practices to nurture
instructional improvements. Feedback to teachers about their professional practice was provided almost
exclusively in a data format and often described as “negative and humiliating.” Interview data revealed that
teachers had a strong sense of commitment to their students and a strong desire to improve instructional practices
to meet student learning needs.

While the school had created structures to provide time for teachers to learn and plan collaboratively, data
brought to PLC meetings by teachers and school leadership were not used to inform instruction. Classroom
observation and interview data confirmed that students had few opportunities to engage in personalized or
differentiated learning tasks. Few students were engaged in rigorous coursework, discussions, or tasks that
required the use of higher order thinking.

Effective, results-driven continuous improvement planning processes with systems, programs, and practices were
not established or used to monitor and communicate results to stakeholders. Further, the school did not routinely
use data to evaluate program effectiveness, monitor the impact of specific strategies, or determine attainment of
school goals. While the Diagnostic Review Team observed some research-based teaching, instruction in many
classrooms lacked rigor and was not grounded in best practices or research. The student performance data
showed that instructional practices were not resulting in increased student achievement. Interview and survey
data and a review of documents and artifacts indicated that teachers and leaders inconsistently engaged in a
continuous improvement and decision-making process to build instructional and organizational capacity. The
established PLC structures could be leveraged to develop, implement, and monitor data-driven processes and
procedures that inform instructional practices and result in higher levels of student achievement.

Next Steps
The results of the Diagnostic Review provide the next step for guiding the improvement journey of the institution
with their efforts to improve the quality of educational opportunities for all learners. The findings are aligned to
research-based criteria designed to improve student learning and organizational effectiveness. The feedback
provided in the Diagnostic Review Report will assist the institution in reflecting on current improvement efforts
and adapting and adjusting their plans to continuously strive for improvement.

Upon receiving the Diagnostic Review Report, the institution is encouraged to implement the following steps:
• Review and share the findings with stakeholders.
• Develop plans to address the Improvement Priorities identified by the Diagnostic Review Team.
• Use the findings and data from the report to guide and strengthen the institution’s continuous improvement
efforts.
• Celebrate the successes noted in the report.

© Advance Education, Inc. 19 www.advanc-ed.org


Diagnostic Review Report

Team Roster
Diagnostic Review Teams comprise professionals with varied backgrounds and professional experiences. All Lead
Evaluators and Diagnostic Review Team members complete AdvancED training and eleot® certification to provide
knowledge and understanding of the AdvancED tools and processes. The following professionals served on the
Diagnostic Review Team:

Team Member Name Brief Biography


Gracie Strawn Gracie Strawn is a semi-retired educator with more than 40 years of varied public
education experience. She has served as a middle school principal, director of
before- and after-school programs, supervisor of K-12 instruction and curriculum,
a regular classroom and special education teacher, and a special education
consulting teacher. She has also trained teachers and administrators at local,
state, and national levels in implementing educational technology and proven
instructional and curriculum practices. While serving as a Tennessee exemplary
educator, she specialized in identifying strengths and weaknesses in school and
system governance and in classroom instruction, and guided a low-performing
high school through a successful turnaround. She holds a bachelor’s degree in
secondary education and special education and a master’s degree in educational
administration and supervision. In addition, she has Reading Recovery training and
certification and career level III certification.
Leesa Moman Leesa Moman is an Education Recovery Leader with the Kentucky Department of
Education (KDE). In that position, she provides support to identified school
districts that have a significant number of schools classified as Targeted Support
and Improvement (TSI). She has extensive experience in assisting districts and
schools as they build systems of continuous improvement, resulting in increased
student academic performance. Ms. Moman holds a bachelor’s degree in
elementary and special education, a master’s degree in special education, and a
Rank 1 in educational administration and supervision. She has also earned
superintendent, principal, supervisor, and director of special education
certifications. Ms. Moman has experience as a teacher, special education teacher
consultant, principal, director of special education, assistant superintendent, and
college professor.
Matthew Courtney Dr. Matthew Courtney currently serves as an Education Program Consultant at the
KDE. In addition to his experience at KDE, Dr. Courtney has teaching experience at
the elementary and collegiate levels. As a teacher, he served on multiple district-
level committees, including district instructional rounds teams. He has taught both
undergraduate and graduate classes in education, leadership, and professional
growth and has been a guest lecturer on topics related to teacher professional
learning and classroom-level leadership. Dr. Courtney has served in leadership and
research roles in the nonprofit sector, specializing in teacher quality and
improvement. He holds a bachelor’s degree in music education from Eastern
Kentucky University, a master’s degree in teacher leadership, and a doctorate
degree in educational leadership from the University of the Cumberlands.

© Advance Education, Inc. 20 www.advanc-ed.org


Diagnostic Review Report

Team Member Name Brief Biography


Paula Goss Paula Goss currently serves as the middle school and high school instructional
coach for the Franklin County School District. She has teaching, coaching, and
consulting experience at all levels K-12 in rural, suburban, and urban settings. Mrs.
Goss’s experience includes being a social studies teacher, an academic consultant,
an academic core branch manager in the division of program standards, an
instructional support specialist, and an instructional coach. Her experience lies in
the areas of curriculum, instruction, and assessment at the local, state, and
national levels. She has provided professional learning experiences for teachers
and administrators and analyzed school and district needs with leadership to raise
achievement. Mrs. Goss holds a bachelor’s degree in middle grade education, a
master’s degree in instructional leadership and a Rank I through National Board
Certification.

© Advance Education, Inc. 21 www.advanc-ed.org


Diagnostic Review Report

Addenda
Student Performance Data
Section I: School and Student Proficiency and Separate Academic Indicator Results
Content Area %P/D School %P/D State %P/D School %P/D State
(16-17) (16-17) (17-18) (17-18)
“All Student Group” “All Student Group”
rd
Reading 3 27.5 55.8 31.2 52.3

Reading 4th 30.3 49.9 34.6 53.7

Reading 5th 33.7 57.3 31.9 57.8

Math 3rd 30.0 50.9 26.0 47.3

Math 4th 32.9 47.9 24.4 47.2

Math 5th 23.6 48.9 37.5 52.0

Science 4th N/A N/A 19.2 30.8

Social Studies 5th 41.6 60.0 33.3 53.0

Writing 5th 14.6 45.9 16.7 40.5

Plus

• The percentage of third-grade students scoring Proficient/Distinguished in reading increased from 27.5
percent in 2016-2017 to 31.2 percent in 2017-2018.
• The percentage of fourth-grade students scoring Proficient/Distinguished in reading increased from 30.3
percent in 2016-2017 to 34.6 percent in 2017-2018.
• The percentage of fifth-grade students scoring Proficient/Distinguished in math increased from 23.6
percent in 2016-2017 to 37.5 percent in 2017-18.
• The percentage of fifth-grade students scoring Proficient/Distinguished in writing increased from 14.6
percent in 2016-2017 to 16.7 percent in 2017-2018.

Delta

• In both 2016-2017 and 2017-2018, the percentage of students scoring Proficient/Distinguished was below
the state average in all content areas and at all grade levels.
• The lowest percentage of students scoring Proficient/Distinguished in 2016-2017 and 2017-2018 was in
fifth-grade writing at 14.6 percent and 16.7 percent, respectively.
• The percentage of scores at the Proficient/Distinguished levels in third-grade math declined from 30
percent in 2016-2017 to 26 percent in 2017-2018.
• The percentage of scores at the Proficient/Distinguished levels in fourth-grade math declined from 32.9
percent in 2016-1017 to 24.4 percent in 2017-2018.
• The percentage of scores at the Proficient/Distinguished levels in fifth-grade social studies declined from
41.6 percent in 2016-2017 to 33.3 percent in 2017-2018.

© Advance Education, Inc. 22 www.advanc-ed.org


Diagnostic Review Report

Section II: Student Growth Index (2017-2018)


Content Area Index State Index

Reading 15.9 19.7

Math 15.1 14.5

EL N/A 31.9

Growth Indicator 15.5 17.1

Plus

• The student growth index in 2017-2018 in math was 15.1, which exceeded the state index of 14.5.

Delta

• The student growth index for 2017-2018 in reading and the growth indicator were below the state index.

Section III: Gap Data for 2017-2018 %P/D


Gap Group Reading Math Science Social Studies Writing
%P/D %P/D %P/D %P/D %P/D

All Students 32.6 29.1 19.2 33.3 16.7


Female 39.4 28.4 23.7 30.6 22.2
Male 26.3 29.7 15.0 36.1 11.1
White 37.8 36.9 34.1 46.9 18.8
African American 22.6 16.7 13.8 10.3
Hispanic 50.0 37.5
Asian
American Indian or
Alaska Native
Native Hawaiian or
Other Pacific Islander
Two or more races 26.7 26.7
Title I 32.6 29.1 19.2 33.3 16.7
Migrant
Homeless
Foster

© Advance Education, Inc. 23 www.advanc-ed.org


Diagnostic Review Report

Gap Group Reading Math Science Social Studies Writing


%P/D %P/D %P/D %P/D %P/D

Military
English Learner (EL)
English Learner plus
Monitored
Economically 29.8 24.5 18.2 29.3 15.5
Disadvantaged
Gifted/Talented
Disability-With IEP 16.0 4.0
(Total)
Disability-With IEP (No 16.0 4.0
Alt)
Disability (no ALT) with
Accommodation
Consolidated Student 26.2 19.7 2.6 22.0 14.6
Group

Plus

• In reading, 39.4 percent of female students and 50 percent of Hispanic students scored
Proficient/Distinguished.
• The percentage of Hispanic students scoring Proficient/Distinguished in math was 37.5.
• The percentage of males scoring Proficient/Distinguished in social studies was 36.1.

Delta

• In writing, the highest percentage of students scoring Proficient/Distinguished in any gap area was less
than 23 percent.
• Four percent of disabled students with IEPs scored Proficient/Distinguished in math.
• In science, 2.6 percent of the consolidated student group scored Proficient/Distinguished.
• In writing, 11.1 percent of male students scored Proficient/Distinguished compared to 22.2 percent of
female students.
• The percentage of African-American students scoring Proficient/Distinguished in writing was 10.3 percent,
as compared to 16.7 percent of all students.

© Advance Education, Inc. 24 www.advanc-ed.org


Diagnostic Review Report

Schedule
Monday, February 11, 2019
Time Event Where Who
4:00 p.m. Team Meeting Hotel Diagnostic
Conference Review Team
Room Members
4:30 p.m.– Principal/Superintendent Presentation Hotel Diagnostic
5:15 p.m. Conference Review Team
Room Members
5:15 p.m.– Team Work Session #1 Hotel Diagnostic
9:00 p.m. Conference Review Team
Room Members

Tuesday, February 12, 2019


Time Event Where Who
8:15 a.m. Team arrives at Johnsontown Road Elementary School School office Diagnostic
Review Team
Members
8:15 a.m. – Interviews / Classroom Observations / Stakeholder Interviews / School Diagnostic
4:15 p.m. Artifact Review Review Team
Members
4:15 p.m. – Team returns to hotel
5:00 p.m.
5:00 p.m. – Team Work Session #2 Hotel Diagnostic
9:00 p.m. Conference Review Team
Room Members

Wednesday, February 13, 2019


Time Event Where Who
8:30 a.m. Team arrives at Johnsontown Road Elementary School School Diagnostic
Review Team
Members
8:30 a.m. – Interviews / Classroom Observations / Stakeholder Interviews / School Diagnostic
4:15 p.m. Artifact Review Review Team
Members
4:15p.m. – Team returns to hotel
5:00 p.m.
5:00 p.m. – Team Work Session #3 Hotel Diagnostic
9:00 p.m. Conference Review Team
Room Members

Thursday, February 14, 2019


Time Event Where Who
8:00 a.m. – Final Team Work Session School Diagnostic
10:30 a.m. Review Team
Members

© Advance Education, Inc. 25 www.advanc-ed.org


advanc-ed.org

Toll Free: 888.41EDNOW (888.413.3669) Global: +1 678.392.2285, ext. 6963


9115 Westside Parkway, Alpharetta, GA 30009

About AdvancED

AdvancED is a non-profit, non-partisan organization serving the largest community of education

professionals in the world. Founded on more than 100 years of work in continuous improvement,

AdvancED combines the knowledge and expertise of a research institute, the skills of a management

consulting firm and the passion of a grassroots movement for educational change to empower

Pre-K-12 schools and school systems to ensure that all learners realize their full potential.

©Advance Education, Inc. AdvancED® grants to the Institution, which is the subject of the Engagement Review Report,
and its designees and stakeholders a non-exclusive, perpetual, irrevocable, royalty-free license, and release to
reproduce, reprint, and distribute this report in accordance with and as protected by the Copyright Laws of the United
States of America and all foreign countries. All other rights not expressly conveyed are reserved by AdvancED.

You might also like