Heather Vogell et al., Atlanta Journal Constitution, March 25, 2012
Suspicious test scores in roughly 200 school districts resemble those that entangled Atlanta in the biggest cheating scandal in American history, an investigation by The Atlanta Journal-Constitution shows.
The newspaper analyzed test results for 69,000 public schools and found high concentrations of suspect math or reading scores in school systems from coast to coast. The findings represent an unprecedented examination of the integrity of school testing.
The analysis doesn’t prove cheating. But it reveals that test scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools.
A tainted and largely unpoliced universe of untrustworthy test results underlies bold changes in education policy, the findings show. The tougher teacher evaluations many states are rolling out, for instance, place more weight than ever on tests.
Perhaps more important, the analysis suggests a broad betrayal of schoolchildren across the nation. As Atlanta learned after cheating was uncovered in half its elementary and middle schools last year, falsified test results deny struggling students access to extra help to which they are entitled, and erode confidence in a vital public institution.
In nine districts, scores careened so unpredictably that the odds of such dramatic shifts occurring without an intervention such as tampering were worse than one in 10 billion.
In Houston, for instance, test results for entire grades of students jumped two, three or more times the amount expected in one year, the analysis shows. When children moved to a new grade the next year, their scores plummeted — a finding that suggests the gains were not due to learning.
Overall, 196 of the nation’s 3,125 largest school districts had enough suspect tests that the odds of the results occurring by chance alone were worse than one in 1,000.
For 33 of those districts, the odds were worse than one in a million.
A few of the districts already face accusations of cheating. But in most, no one has challenged the scores in a broad, public way.
The newspaper’s analysis suggests that tens of thousands of children may have been harmed by inflated scores that could have precluded tutoring or more drastic administrative actions.
The analysis shows that in 2010 alone, the grade-wide reading scores of 24,618 children nationwide — enough to populate a midsized school district — swung so improbably that the odds of it happening by chance were less than one in 10,000.
The findings come as government officials, reeling from recent scandals, are beginning to acknowledge that a troubling amount of score manipulation occurs. Though the federal government requires the tests, it has not mandated screening scores for anomalies or investigating those that turn up.
Test-score pressure is palpable in schools grappling with urban blight and poverty.
These are the schools that the 2001 No Child Left Behind Act was supposed to fix.
But at Patrick Henry Downtown Academy in St. Louis, airy red brick towers rising above the school belie a grimmer reality on the ground. Children leaving one recent afternoon passed piles of trash and a .45 caliber bullet tucked into the curb. Inside, their classrooms are beset by mold, rats, discipline problems and scandal.
Last year, the former principal — once hailed as among the district’s strongest — was accused by Missouri officials of falsifying attendance rolls to get more state money.
State investigators didn’t publicly question Henry’s test scores.
But the AJC’s analysis found suspicious scores in the school dating back to 2007. In 2010, for instance, about 42 percent of fourth-graders passed the state math test. When the class took the tests as fifth-graders the next year — with state investigators looking into cheating and other fraud allegations — just 4 percent passed math.
Experts say student learning doesn’t typically jump backwards.
Henry’s scores were consistently among the lowest in the state — except for the occasional sudden leap.
Big-to-medium-sized cities and rural districts harbored the highest concentrations of suspect tests. No Child Left Behind may help explain why. The law forced districts to contend with the scores of poor and minority students in an unprecedented way, judging schools by the performance of such “subgroups” as well as by overall achievement.
Hence, high-poverty schools faced some of the most relentless pressure of the kind critics say increases cheating.
Improbable scores were twice as likely to appear in charter schools as regular schools. Charters, which receive public money, can face intense pressure as supposed laboratories of innovation that, in theory, live or die by their academic performance.
Common problems unite the big-city districts with the most prevalent suspicious scores: Many faced state takeovers if scores didn’t improve quickly. Teachers’ pay or even their continued employment sometimes depended on test performance. And their students — mostly poor, mostly minority — were among those needing the most help.
Some of the most persistently suspicious test scores nationwide, however, occurred in districts renowned for cutting-edge reforms.
In Atlanta, for instance, former Superintendent Beverly Hall won national recognition as Superintendent of the Year in 2009. State investigators later confirmed scores that year were widely manipulated by educators who assisted students improperly and outright changed tens of thousands of their answers on state tests.
In some Atlanta schools, cheating was an open secret for years. After students turned in their tests, teachers and administrators erased and corrected their mistakes — even holding a “changing party” at a teacher’s home. In another school, staff opened plastic wrap securing test booklets with a razor, then melted the wrap shut again after making forbidden copies.
State investigators accused a total of 38 principals with participating in test-tampering. One allegedly wore gloves while erasing to avoid leaving fingerprints.
In 2002, [the city of] Houston was the first winner of the Broad Prize, which has become the most coveted award in urban education. The Eli and Edythe Broad Foundation praised Houston’s intense focus on test results. More recently, Houston has been among the leaders in tying teacher pay to student test scores.
But twice in the past seven years, the AJC found, Houston exhibited fluctuations with virtually no chance of occurring except through tampering.
In 2005, scores fell precipitously in five dozen classes in 38 schools after a statistical analysis by the Dallas Morning News suggested test-tampering in Houston. The district fired teachers and principals and improved test security.
In 2011, however, as three-fourths of Houston teachers earned performance-based bonuses, scores rose improbably in a similar number of classes in the same number of schools. In the same year, Houston confirmed nine cheating allegations and fired or took other action against 21 employees.
The AJC sent detailed findings to districts with some of the most suspicious clusters of scores. For those not already publicly looking at cheating, the responses were similar: Officials said they were unaware of most anomalies, but protested characterizing the score changes as cheating.
Some districts simply denied any problems exist. Detroit, for instance, claimed its scores were not “unusual or out of line in any way” and that Michigan officials had not identified irregularities “with respect to an erasure analysis, suspected cheating, or any other issue.”
Through programs such as Race to the Top, federal education officials have pushed states to adopt more aggressive teacher evaluation systems that, typically, consider test scores.
“Whatever the stakes were under No Child Left Behind,” Ravitch said, “they are going to be much higher, now that teachers are being told your scores are going to be public and you’re going to be fired if they don’t go up X number of years in a row.”