How Mississippi gamed its national reading test scores to produce ‘miracle’ gains

by Michael Hiltzik for the Los Angeles Times

The term has been shooting around the education field and news reports lately with increasing frequency: “the Mississippi miracle.”

The reference is to that benighted state’s surprising success in improving reading scores for its fourth-graders through a focused program of literacy instruction for teachers and pupils alike. It’s now 10 years old, an anniversary that may have inspired the most recent assessments.

Statistics show that Mississippi’s children have gone from having almost the worst scores on the standardized national reading test for fourth-graders in 2013 to narrowly exceeding the national average in the most recent test, administered last year.

The turnaround there (and smaller gains in Louisiana and Alabama, which enacted similar literacy programs) “has grabbed the attention of educators nationally, showing rapid progress is possible anywhere, even in areas that have struggled for decades with poverty and dismal literacy rates.”

Education writers and the New York Times jumped on the bandwagon. (“Mississippi Is Offering Lessons for America on Education,” was the latter’s headline.)

As an old journalism adage has it, the story is “interesting, if true.”

A close examination of the numbers suggests that it’s not true. Bob Somerby and Kevin Drum, two of the most adept mythbusters in the blogosphere, have done yeomen’s work deconstructing the statistics. Their conclusion is that Mississippi’s program isn’t nearly as successful as its fans assert and may not have produced any improvement at all in fourth-grade reading scores. The apparent gains may be a statistical illusion…

…What’s the real story? Drum and Somerby focused on the so-called “third-grade gate” implemented by the literacy program — the requirement that third-grade underachievers repeat third grade. In Mississippi, almost 10% of third-graders have been getting held back, a higher proportion than any other state. (Some may have been held back more than once.)

The statistical result of this policy should be obvious. If you throw the lowest-ranking 10% out of a statistical pool, the remaining pool inevitably looks better. Drum went so far as to add those dropped pupils back into the calculation. He found that the gains from 2013 to 2022 completely disappeared. “In other words,” he remarked, “the 2013 reforms had all but no effect.”

Read more here.