Are New York State Common Core Grades 3-8 Tests Valid,

Reliable and Stable Reflectors of Student Achievement?

BLOG Posted on August 13, 2014 | 2 Comments

Are New York State grades 3-8 tests valid, reliable and stable reflectors of student
achievement? In other words, are these “good” tests? And, are they useful to parents and
teachers, or, just poorly designed accountability tools to measure teacher, principal and school
performance?

On the eve of the release of the 2014 ELA/Math test results parents and teachers are wary.

Last year, the first year of the common core tests, the state tried, unsuccessfully, to mollify
parents. The commissioner tells us, yes, your kids did terribly, swallow the medicine, in the long
run your kids will be better prepared for college and life.

The 2013 State presser:

“These proficiency scores do not reflect a drop in performance, but rather a raising of standards
to reflect college and career readiness in the 21st century,” King said. “I understand these
scores are sobering for parents, teachers, and principals. It’s frustrating to see our children
struggle. But we can’t allow ourselves to be paralyzed by frustration; we must be energized by
this opportunity. The results we’ve announced today are not a critique of past efforts; they’re a
new starting point on a roadmap to future success.

• 31.1% of grade 3-8 students across the State met or exceeded the ELA proficiency
standard; 31% met or exceeded the math proficiency standard
• The ELA proficiency results for race/ethnicity groups across grades 3-8 reveal the
persistence of the achievement gap: only 16.1% of African-American students and 17.7% of
Hispanic students met or exceeded the proficiency standard
• 3.2% of English Language Learners (ELLs) in grades 3-8 met or exceeded the ELA
proficiency standard; 9.8% of ELLs met or exceeded the math proficiency standard
• 5% of students with disabilities met or exceeded the ELA proficiency standard; 7% of
students with disabilities met or exceeded the math proficiency standard

As the parent anger grew the commissioner embarked on a listening tour – at the first stop in
Poughkeepsie parents responded angrily. See U-Tube of meeting:
(https://www.youtube.com/watch?v=P_Eiz406VAs)

The rumors are the scores across the state this year are “flat.” Over 30,000 parents in the state
opted-out of the state tests and the parent revolt around the state morphed from a brush fire to a
conflagration. At the end of the legislative session a law stayed the impact of student test scores
on students and teachers for two years. The Republicans have just filed petitions, with over
60,000 signatures to add a “Stop the Common Core” line to the November gubernatorial ballot.

Annual state tests only began in 2003 with the passage of a reauthorization of the 1965
Elementary and Secondary Education Act (ESEA), renamed, No Child Left Behind (NCLB). The
law required the testing of all students in grades 3 through 8 (previously only students in grade 4
and 8 were tested), the setting of a progress goal, called Adequate Yearly Progress (See
detailed description here), the public reporting of scores by sub-group with the goal of having all
student achieve on grade level by 2014.

Each year the state tests were administered in April, all the tests questions released a few
months later, the student scores released in August and a Technical Report released in
December, a detailed analysis of the test.

Beginning with the 2013 common core tests the state changed the practice – only 25% of the
questions were released and an abbreviated Technical Report was not issued until July 2014,
fifteen months after the exam was administered; for the 2014 round of tests the state has just
released 50% of the test questions.

Fred Smith, a former testing guru in the NYC Department of Education raises the cogent
questions (Read entire article here),

… the State Education Department took a half-step by releasing 50 percent of the English and
math questions from the April 2014 exams. It was a half-step not just because it falls halfway
short of full disclosure, but also because SED fails to provide data at its disposal that would
enable objective evaluation of the questions, each of which is a brick in the wall of the testing
program.

Until 2011, SED made complete copies of the annual statewide exams available on its web site
shortly after they were given, along with the answer keys. People could ponder the test content
and thought processes required of students to answer the questions. Every year, no later than
December, a technical report was issued giving specific information about each item.

These data included the percentage of students who answered the item correctly (the p-value)
and the percentage who chose each incorrect option (distractor) or omitted the item. In addition,
a vital statistic was reported showing how well the item functioned (the discrimination index).
This allowed reviewers to learn about how every item was working, to identify which ones might
be weak and to reach an overall judgment about test quality.

The ability to see all operational items in a timely manner and match them one-to-one against
analytic data afforded a reasonable way to study the tests. Since Pearson became the test
publisher in 2011, however, SED has hoarded the kind of item-level data needed to render an
informed opinion about the tests.

The state has never given a satisfactory answer to the crucial question: why they closed the
door to full transparency? At one point claiming the cost of releasing all the questions is too
high, or, due to the nature of the contract with Pearson there are confidentiality issues, are not
acceptable. The standards setting, the process by which each question is assigned a difficulty
level (Level 1-4) is also cloaked in secrecy – the members of the standards setting team must
sign confidentiality agreements, principals and teachers are banned from discussing individual
questions; secrecy only leads to speculation: does Pearson or the state have something to
hide?

In the pre-2013 era the tests reflected curricula content – to what extent did students “learn” the
particular content in ELA and math that was required for each grade, the common core exams
were significantly different, rather than testing content the exams test the standards – to be
more accurate the application of the standards. Schools are now expected to teach content
through the lens of specific standards set by distant anonymous committees under the direction
of the National Governors Association, and, the actual curriculum modules have been released
piecemeal by the state and vary in quality. The state maintains curriculum decisions should be
made at the local level.

If the purpose of the tests are to measure student progress, sampling techniques, the
procedures used by NAEP. PISA and TIMMS, would be sufficient. If the purpose is to inform
parents and teachers of student weakness the tests do not achieve that purpose. The sole
purpose of the exams appears to be to hold teachers, principals and schools accountable. The
state maintains the 2013 tests set a baseline and each year the results measure the growth, or
lack thereof, from the baseline.

Whether an improvement in scores is due to test sophistication or improved instruction is

impossible to tell. It will be years before we can determine whether the common core standards
have any impact on student learning.

Unfortunately the mishandling of the rollout of the tests, the refusal to delay the implementation,
the ragged release of curriculum modules, and especially an arrogance toward parent concerns
have thrust the common core into the political arena.

Peter DeWitt, a former New York State principal, writes in Education Week,

We should stop talking about achievement and start having real dialogue about growth.
Achievement means that leaders like King only care about the adults in the room (teacher
evaluation), and his lack of experience in the classroom doesn’t help him understand the
importance of getting students to become assessment capable (Hattie. 2012).

Focusing on growth will help students become assessment-capable, college and career ready,
21st century learners. Until King discusses growth and not achievement, and can provide
schools with feedback that is “Just in time and just for them in a timely manner that matters
most (Hattie)” he will just be sending out another memo that infuriates school leaders and gets
deleted in much quicker fashion than it took to write.

Unless we radically change the testing process in New York State the winds of politics will
prevail and the common core will become another failed innovation.