When Standardized Test Scores Soared in D.C., Were the Gains Real?

In just two years, Crosby S. Noyes Education Campus went from a school deemed in need of improvement to a place that the District of Columbia Public Schools called one of its “shining stars.”

{snip}

Standardized test scores improved dramatically. In 2006, only 10% of Noyes’ students scored “proficient” or “advanced” in math on the standardized tests required by the federal No Child Left Behind law. Two years later, 58% achieved that level. The school showed similar gains in reading.

Because of the remarkable turnaround, the U.S. Department of Education named the school in northeast Washington a National Blue Ribbon School. Noyes was one of 264 public schools nationwide given that award in 2009.

Michelle Rhee, then chancellor of D.C. schools, took a special interest in Noyes. She touted the school, which now serves preschoolers through eighth-graders, as an example of how the sweeping changes she championed could transform even the lowest-performing Washington schools. Twice in three years, she rewarded Noyes’ staff for boosting scores: In 2008 and again in 2010, each teacher won an $8,000 bonus, and the principal won $10,000.

A closer look at Noyes, however, raises questions about its test scores from 2006 to 2010. Its proficiency rates rose at a much faster rate than the average for D.C. schools. Then, in 2010, when scores dipped for most of the district’s elementary schools, Noyes’ proficiency rates fell further than average.

A USA TODAY investigation, based on documents and data secured under D.C.’s Freedom of Information Act, found that for the past three school years most of Noyes’ classrooms had extraordinarily high numbers of erasures on standardized tests. The consistent pattern was that wrong answers were erased and changed to right ones.

Noyes is one of 103 public schools here that have had erasure rates that surpassed D.C. averages at least once since 2008. That’s more than half of D.C. schools.

Erasures are detected by the same electronic scanners that CTB/McGraw-Hill, D.C.’s testing company, uses to score the tests. When test-takers change answers, they erase penciled-in bubble marks that leave behind a smudge; the machines tally the erasures as well as the new answers for each student.

In 2007-08, six classrooms out of the eight taking tests at Noyes were flagged by McGraw-Hill because of high wrong-to-right erasure rates. The pattern was repeated in the 2008-09 and 2009-10 school years, when 80% of Noyes classrooms were flagged by McGraw-Hill.

On the 2009 reading test, for example, seventh-graders in one Noyes classroom averaged 12.7 wrong-to-right erasures per student on answer sheets; the average for seventh-graders in all D.C. schools on that test was less than 1. The odds are better for winning the Powerball grand prize than having that many erasures by chance, according to statisticians consulted by USA TODAY.

{snip}

A trio of academicians consulted by USA TODAY–Haladyna, George Shambaugh of Georgetown University and Gary Miron of Western Michigan University–say the erasure rates found at Noyes and at other D.C. public schools are so statistically rare, and yet showed up in so many classrooms, that they should be examined thoroughly.

USA TODAY examined testing irregularities in the District of Columbia’s public schools because, under Rhee, the system became a national symbol of what high expectations and effective teaching could accomplish. Federal money also was at play: Last year, D.C. won an extra $75 million for public and charter schools in the U.S. government’s Race to the Top competition. Test scores were a factor.

USA TODAY initially looked at Noyes only because of its high erasure rates. Later, the newspaper found that Wayne Ryan, the principal from 2001 to 2010, and the school had been touted as models by district officials. They were the centerpiece of the school system’s recruitment ads in 2008 and 2009, including at least two placed in Principal magazine.

{snip}

There can be innocent reasons for multiple erasures. A student can lose his place on the answer sheet, fill in answers on the wrong rows, then change them when he realizes his mistake. And, as McGraw-Hill said in a March 2009 report to D.C. officials, studies also show that test-takers change answers more often when they are encouraged to review their work. The same report emphasizes that educators “should not draw conclusions about cheating behavior” from the data alone.

Haladyna notes, however, that when entire classrooms at schools with statistically rare erasures show fast-rising test scores, that suggests someone might have “tampered with the answer sheets,” perhaps after the tests were collected from students. Although not proof of cheating, such a case underscores the need for an investigation, he says.

At Noyes, USA TODAY found several grades with wide swings in their proficiency rates from one year to the next. In 2008, 84% of fourth-grade math students were listed as proficient or advanced, up from 22% for the previous fourth-grade class. The math scores for the fourth-grade class in 2010 dropped off to 52% proficient or better.

For the school as a whole, test scores seemed to ride a roller coaster: In reading, from 2006 through 2010, the annual percentage of all Noyes students testing as proficient or higher went from 24% to 44% to 62% to 84% to 61%, according to official records. Reading scores at all D.C. elementary schools slumped on average by 4 percentage points from 2009 to 2010; Noyes’ scores plunged 23 points.

{snip}