To Gauge Success, We Need to Look Beyond Third Grade

A rust belt district has seen remarkable reading scores, but it's not clear how long students continue to benefit.

---

The latest installments of the “Sold a Story” podcast juggernaut celebrate a depressed small city in Ohio for its astounding success in teaching kids to read. But does that early success carry over to high school and beyond?

It’s clear that there’s much to admire about the way Steubenville is implementing Success for All, an elementary literacy program that includes structured, explicit instruction in foundational reading skills like phonics. For nearly 20 years, we learn in one of three new episodes, 93 percent or more of Steubenville’s third graders have scored proficient on state reading tests. The city even shines in the national context, scoring in the top 10% of districts across the country—again, for third-grade reading.

But what happens past third grade? “Sold a Story” cites evidence that students who get Success for All are still ahead of their peers at the end of eighth grade. And by some measures, Steubenville does look pretty good at higher grade levels.

According to US News, 72 percent of the city’s high school students and 62 percent of its middle school students score proficient in reading on state tests, along with 87 percent of elementary students. The state report card gives the district five stars in all categories, indicating that it significantly exceeds state standards. That’s impressive, especially for a city where nearly 80 percent of students are poor enough to quality for free or reduced school meals.

At the same time, though, the Educational Opportunity Project at Stanford—the same entity that puts Steubenville in the top 10 percent of districts nationwide—finds that students’ academic growth, as measured by improvement in test scores, is lower than the U.S. average. In fact, the project reports that students in the district learn about 20 percent less each grade than the U.S. average and 13 percent less than students in districts with similar demographics.

Maybe it’s hard for a district like Steubenville to show growth because its third graders are already performing at a high level, so there isn’t much room for them to grow. Or maybe something else is going on.

What Happens in High School and Beyond

Writing in The74, Chad Aldeman notes that Steubenville’s high test scores “have not guaranteed a path to economic security” for its students. “Despite its near-perfect early reading scores, strong middle and high school achievement and a 96% graduation rate,” he writes, “the district’s post-high school results are only slightly above statewide averages in terms of college-going and completion rates and the percentage of graduates who find ‘gainful employment.’”

That could be, as Aldeman suggests, a function of the depressed economic climate in the community rather than “anything that the school district itself directly controls.” But there are reasons to believe that the district could be doing more to enable its students to fully capitalize on their early success.

While Steubenville’s high school students do well on state reading tests, statistics available on the state report card website show that only about 7 percent scored high enough on the ACT or SAT to be considered ready for college-level coursework. And none got an AP score of three or better—the threshold for college credit. Only about 27 percent of the graduating class of 2017 graduated from college within six years, compared to a state average of about 30 percent.

I haven’t been to Steubenville, and it’s hard to know what’s really going on based on test scores and college graduation rates. One commentator, writing recently in The74, speculates that the school district isn’t providing enough non-academic support. Maybe students need to be made more aware of career possibilities beginning in middle school, or better advising in high school, or “persistence coaches” through the first year of college.

No doubt all those things could benefit Steubenville’s students. But another possibility, based on the district’s SAT and ACT scores, is that many students are graduating from high school without the academic skills and knowledge that would enable them to succeed. Not every high school graduate needs to go on to college, but given the high test scores Steubenville’s kids achieve at lower grades, you’d expect more than 7 percent to be ready for college-level work.

And the ability to understand complex text is valuable in life as well as college. Adults need to be able to understand a lease agreement, a job training manual, a newspaper article—all of which can have complex vocabulary and syntax. To equip students for those tasks—especially students who come from less highly educated families, a category that is likely to include most Steubenville students—it’s crucial to start early.

Does Success for All Build Knowledge?

Which brings us back to Success for All. I haven’t had access to the program itself, but I’ve spoken with a former administrator in a district that used it for years, and I’ve tried to glean as much as I can from the website of the Success for All Foundation.

According to the website, students are taught “comprehension monitoring and strategies” and they get explicit vocabulary instruction. In addition, a “variety of texts … are taught and are used to develop students’ background knowledge.”

But are the texts grouped so that students spend at least two or three weeks diving into a topic in a way that enables them to absorb and retain new knowledge and vocabulary? Or is the “variety of texts” random, so that students are reading about the solar system one day and dinosaurs the next?

Is the curriculum organized by comprehension skills, with texts used primarily as vehicles to teach the skills? Or does it put content in the foreground, bringing in skills as appropriate to a particular topic or text? Do all students have access to complex text through whole-class read-alouds and discussion, or are they limited to “leveled” texts they can theoretically read on their own?

“Sold a Story” informs us that “a lot of writing” is built into Success for All instruction—and it seems that it’s related to what students are reading. In a first-grade classroom, we hear, students read a story “about a boy who’s trying to get rid of his hiccups” and then write about it. That’s great. There’s evidence that having students write about texts they’re reading improves their comprehension.

But are students also explicitly taught how to construct complex sentences and create linear outlines for paragraphs and essays, so that they’re not cognitively overwhelmed by the demanding task of writing? Are they writing not just about stories but also about social studies and science content in a way that reinforces their knowledge and deepens their comprehension?

Maybe—but I didn’t hear anything about that on “Sold a Story,” and there’s no information about those aspects of the program, if they exist, on the Success for All Foundation website. It’s hard to find much information elsewhere. Success for All hasn’t been evaluated by the organizations that rate reading curricula, EdReports and The Reading League.¹ Nor is it listed on the website of the Knowledge Matters Campaign, which identifies curricula that are effective in building knowledge. (I serve on the board of the nonprofit that is the parent organization of the Knowledge Matters Campaign.)

None of this is to detract from what Success for All does do. It’s apparently quite good at teaching students to decode words, which is crucial. It’s just not enough to guarantee academic success at higher levels, or success in life, especially for kids whose families aren’t in a position to provide them with much academic knowledge or help with writing outside of school.

Because of the narrow focus on problems with phonics instruction over the past several years, there’s now a widespread assumption that if we just “fix phonics,” all our literacy problems will be solved. Unfortunately, it’s going to take a lot more than that to enable all students to develop to their full potential and become truly literate adults.

Some may argue that it’s too much to expect schools to teach kids to decode words, build their academic knowledge, and explicitly teach them to write all at the same time. But the fact is, it’s not only possible to do all those things at once, it actually makes learning easier: there are synergies between all these aspects of literacy. It can make it easier to teach as well, especially if teachers can find all those elements in one curriculum instead of having to juggle several different programs.

If we want to know what really works in education, we need to look at outcomes well beyond third grade or even eighth grade test scores—and scrutinize elementary curricula for more than their approach to foundational reading skills.

1. In the final episode of the Steubenville series, “Sold a Story” does detail some of the problems with EdReports, whose ratings are accepted by many districts as guarantees of quality. I’ve also noted problems with EdReports in this post.