Testing, Testing

What's behind historically low reading and math scores for 12th graders?

Photo by kyo azuma on Unsplash

---

I wasn’t planning to write about the recent release of NAEP scores for 12^th grade reading and math. NAEP, for the uninitiated, stands for the National Assessment of Educational Progress, and the perennial story is that there hasn’t been any progress. That’s important, but it’s become a “dog bites man” story, and I feel I’ve said pretty much all I have to say about it. For those who haven’t read what I’ve said previously, here’s a footnote summarizing my take (and that of some others), on standardized reading comprehension tests.¹

Still, it’s worth mentioning that this time around, 12^th grade scores hit historic lows, with about a third of students scoring below Basic in reading and 45 percent scoring below that level in math. What does that mean? We don’t really know.

NAEP guidelines say that 12^th graders at the Basic reading level “likely can locate and identify relevant details in the text in order to support literal comprehension.” In math, they “likely can determine probabilities of simple events from 2-way tables and verbal descriptions.” So we can infer that students who score below that level are unable to do those things. But they might also struggle just to decipher the words in the test passages or to multiply numbers.²

The common wisdom is that the explanation is the pandemic, but a few astute observers have pointed out that scores were declining and gaps were widening before the pandemic. Those observers include Eric Hanushek, Marty West, and Chad Aldeman—and, in the mainstream media, Jessica Grose of the New York Times. Grose quotes a NAEP insider, Lesley Muldoon, as saying that the steady decline in reading and math started in 2013.

That’s an important point because it indicates that the root causes of our lack of progress go deeper than the disruptive effects of Covid. There’s some uncertainty about what those root causes are, but most commentators are settling on “phones” (as a shorthand for smartphones, social media, and screens generally) and too little test-based accountability.

My take is that these commentators are right about the effects of phones—although maybe not, as most assume, because they lead to declining attention spans. Cognitive scientist Dan Willingham has argued in Education Next that the real problem may not be that kids can’t pay attention; it’s that they don’t want to. Their devices provide the kind of constant stimulation that teachers and books can’t compete with. Schools, Willingham suggests, need to show students that sustained attention can bring long-term rewards. Plus, they could ban phones, eliminating the temptation to pursue immediate gratification.

That makes sense to me. What doesn’t make sense is the idea that bringing back—or increasing—test-based accountability will increase reading test scores, at least the way reading tests are currently constituted. Holding schools and educators accountable for test-score results can only work if the tests are assessing something truly measurable. And comprehension isn’t measurable in the abstract.

We did already try federally enforced test-based accountability, from 2002 to about 2011, under No Child Left Behind, and it didn’t work, at least for reading. Yes, reading scores rose a little before 2011, but they’ve been essentially stagnant since 1998. Plus, most states are still holding schools accountable for test scores, even if they’ve softened the consequences for poor performance and changed the formula to measure not just scores but “growth.”

In any event, the idea that schools need to drill kids on the abstract comprehension skills tests purport to measure is by now so ingrained that most educators are going to do it even if no one is looking over their shoulders. More “accountability” would only exacerbate that problem.

Going Deeper into Falling Scores

The only commentary I’ve seen on the NAEP scores that focuses on curriculum and instruction (while also noting that the decline started before the pandemic) comes from Holly Korbey and relates to math. (It appears on her excellent Substack, The Bell Ringer.) She points to a shortage of secondary math teachers; problems with teacher training; a decrease in homework; a lack of support for struggling students; and—crucially—a divergence between typical teaching techniques and those supported by evidence.

Most of those factors apply to literacy as well, although reading comprehension instruction also has a fundamental lack-of-content problem. The principles of evidence-based teaching, and the consequences of diverging from them, apply across subject areas. And it’s not like there’s a firewall between literacy and math. If you can’t read and understand a word problem, you’re unlikely to do well on a 12^th grade math test.

The decline in NAEP scores over the years has been driven by lower scores for the lowest-performing students, and as Aldeman notes, that continues to be true. But there may be some new twists.

Brandon Wright says the top students are struggling too, with declines in reading at that level attributable mostly to girls and students from middle-class or wealthier families. And while children of highly educated parents still do a lot better than others, Grose notes that nearly 40 percent of the “lower performers” have parents who graduated from college. Over 40 percent are from the middle of the socioeconomic distribution—about the same percentage as came from the low end.³

It's not clear why that’s happening—maybe phones. But the silver lining is that if the decline in scores is now hitting families with more resources and political clout, maybe we’ll hear more of an outcry about the situation.

Still, we’ll have to do more than ban phones if we want to enable all kids to become fully literate. Grose describes how phone bans in some Kentucky schools have led to a dramatic increase in kids checking books out of the library—including at a school where only 17 percent of students scored proficient in reading on state tests. That’s a positive development, but if you’re struggling to read, you’re unlikely to read for fun even if the school takes your phone away—or at least, unlikely to read anything resembling complex text.

It's also long been true that it’s easier to boost 4^th grade reading scores than 8^th or 12^th grade scores, basically because the test passages get harder to decode and understand. Along those lines, Mike Petrilli, whose new Substack alerted me to several of the articles I’m citing here, notes that the 12^th-graders whose reading scores just hit a historic low are members of the same generation that had record high reading scores in 4^th grade. And, once again, this wasn’t just “because pandemic.” Scores had already declined significantly for that cohort between their 4^th and 8^th grade years. Petrilli concludes that the culprit is … phones.

Okay, yes, but maybe in addition, something has been missing from our approach to teaching comprehension. Reading scores at lower grade levels are more dependent on the ability to decode words, while those at higher levels depend on that and the ability to comprehend more complex text.⁴ Even if schools aren’t solely responsible for students’ declining reading ability, as they surely are not, they are in a unique position to address it.

What’s Happening with State Reading Tests?

In addition to the NAEP, which is low-stakes, there are also state reading and math tests, which are considered high-stakes—even if they’re not as high-stakes as commentators like Hanushek, Aldeman, and West would like. One state, Louisiana, has experimented with a change that would make its reading tests fairer and more accurate: grounding the passages in content students have actually been taught in school rather than using random topics. Louisiana was able to do that because many schools there use its state-created literacy curriculum, which focuses on content rather than abstract comprehension skills.

One advantage of that approach to testing is that instead of dropping instruction to focus on test prep, teachers can prepare kids for the tests by continuing to teach the content in the curriculum. Another is that feedback from the tests, which are given as three end-of-unit assessments throughout the year, can help teachers accurately determine what students have and haven’t learned.

Educators in one district that has participated in the pilot, which started in 2020, told me students who take the new tests are more engaged and more confident. Teachers appreciate the feedback they get. And test scores have risen, although the new test may not be the only reason.

But uptake across the state has been slow, apparently because of hesitation to try something new. As of the 2023-24 school year, only 28 percent of Louisiana districts were participating in the pilot, which only covers grades six through eight.

Now, unfortunately, the pilot has been terminated, according to someone who was involved in the design of the new tests: Dr. David Steiner, executive director of the Institute for Education Policy at Johns Hopkins University. Steiner explained that federal regulations required that at the end of the pilot period, the new assessment had to be used state-wide in order to continue. By that time, the state had also created a reading test tied to the content in another curriculum, Wit & Wisdom. Over 85 percent of students in the state taking English language arts used either that curriculum or that state-created one, called ELA Guidebooks. But that still left around 15 percent who didn’t, so it wasn’t possible to use the curriculum statewide.

Other states are in an even worse position to try Louisiana’s experiment because their schools are using a plethora of different literacy curricula, some of which may not have much content at all. But there’s something else they—and now Louisiana—could easily do: ground reading test passages in the content specified by their social studies and science standards. All states have those content standards, and many of them aren’t great—especially in social studies. But at least—unlike state literacy standards, which are just lists of skills—they specify some content.⁵

I’ve heard that at least a couple of states, including Texas, are considering doing that. Texas is switching from having one round of state testing at the end of the year to administering three shorter tests throughout the year, as in the Louisiana pilot. But if the tests aren’t grounded in specific content, they’ll just add to the time spent on testing without yielding benefits.

Schools Aren’t Assigning Whole Books

One reason for stagnant and declining reading scores is a misguided focus on abstract comprehension skills (as opposed to teaching whatever skills make sense for a particular text). Another related one is schools’ reliance on short texts and brief excerpts as opposed to whole books. Kids might read the first chapter or two of a novel but never see the rest.

Why that’s happening and what the consequences are is a big topic and one that I’ll be addressing in the Winter issue of American Educator.⁶ You can also read an impassioned argument for reading whole books in the new book The Teach Like a Champion Guide to the Science of Reading by Doug Lemov, Colleen Driggs, and Erica Woolway (they’ve also created a book-centric middle grades ELA curriculum). But for now I’ll just highlight an explanation for the brief-text phenomenon that many might overlook.

On the Curriculum Insight Substack, Karen Vaites points out that using brief texts is good for curriculum publishers’ profit margins. At the elementary and middle school level, the most widely used literacy curricula are anthologies. To include whole books, publishers would need to buy the books or pay a lot to incorporate them. It’s cheaper to just hire writers to produce content specifically for the anthology or pay a lower amount for an excerpt.

The ones who are paying the price for this cost-saving maneuver are America’s students.

Update, September 20: This post was changed to include the information that Louisiana’s pilot program had ended.

1. Reading comprehension tests purport to measure general skills like “finding the main idea.” That has exacerbated the practice of trying to teach those skills in isolation from any particular content, which has meant most kids get very little social studies and science. But those marginalized subjects are most likely to build the knowledge that fuels reading comprehension, especially at higher grade levels. So until we shift to a system that systematically builds students’ academic knowledge beginning in the early grades, reading scores will remain low. Also, the gap between kids from more and less highly educated families will continue to widen because kids from more highly educated families are better able to pick up academic knowledge outside school.

2. As Chad Aldeman has argued in a recent article, and as I’ve argued previously, it would help if we at least tested decoding ability separately from comprehension. We know there are many high school students who struggle to decode words, especially multisyllabic ones, so it would make sense to do that at higher as well as lower grade levels. Maybe we should also test students at higher grade levels separately on basic math.

3. Grose seems to be defining “lower performers” as the bottom 25 percent.

4. The high 4^th grade scores also suggest that the cohort in question, or much of it, did get adequate instruction in decoding skills like phonics even before “Sold a Story” came along to expose widespread problems in that area.

5. The NAEP could do a version of this as well by grounding reading passages in the content covered by its infrequently given tests in subject areas like US history, civics, and science.

6. One reason it’s happening is the spread of grading policies that make it optional for students to do homework, which you can read more about in a recent report from the Fordham Institute. (TL;DR: Most teachers think this kind of policy is harmful to academic engagement.) It’s hard to teach a novel if students have to do all the reading in class.]