As the father of three children (all now in college), I find it shocking that this must even be argued. People will be more engaged with a text on a topic they understand, using words they understand. Those who claim that "strategies" are paramount seem to regard humans as mere computers processesing information. What a shame that any such persons are involved in education.
Great article. I find that literacy teachers forget what is fundamental about words: that they are symbols that represent reality. The skills to decode symbols are meaningless without knowledge of the underlying reality they point to.
I observe the phenomenon mentioned here in my own reading. If I research a topic online that has specialized terminology (scientific, for instance) there's a tipping point at which there are too many unfamiliar terms for me to really comprehend a passage. My "working memory" becomes overwhelmed. To understand a passage and make inferences about words you don't know there have to be a sufficient amount of words you do understand.
And besides the question of making inferences, if it is too hard to comprehend the passage you aren't going to experiencing the reward of gaining comprehension so your motivation decreases.
Might I suggest researchers start looking at how countries with high literacy and comprehensive skill rates educate their children. Seems like the practical thing to do.
Some researchers have done that, although trying to transplant practices that work in one country to another country is tricky. There can be huge differences in context and culture. Generally, however, the research has found that countries with a clearly defined national curriculum and tests aligned to the content of that curriculum do better in international comparisons.
Education in the U.S. is highly localized, so there's no possibility of a national curriculum here. No state even has a mandatory curriculum, although legally they could. But there's a lot that can be done at the state level to encourage adoption of content-rich, knowledge-building curricula. And states could ground the passages on reading tests in their content standards in science and social studies.
Oh, I totally understand that but looking at other successful methods could generate useful ideas to incorporate into discussions. It sounds like people are talking past each other instead of to each other. Proof of a working concept could help with that.
That sort of bothers me that we don't have a national curriculum. Children in red states are being short changed because of it. For instance, when I was growing up in Ohio I got a first class public education. Fast forward 40 years and education isn't that great in Ohio anymore.
That sounds like a great idea! Having even a little background knowledge in a subject is very helpful for comprehension. I used to read complex medical studies just for fun, lol. Even though I'm not a doctor I understood quite a lot with the little background knowledge I do have.
Thank you for posting another thought provoking article.
Folks, the study says exactly nothing about the relative importance of background knowledge and reading skills. The authors wanted to compare good and poor readers but didn’t succeed in doing so. There are no effects of reader group in this study. The procedures used to select participants did not yield groups that differed in reading skill. The study therefore does not show anything about the effect of reading skill or the interaction of reading skill and prior knowledge. The experiment would have yielded the same results if the texts had been read aloud to the participants–if there were no reading involved at all.
The authors made the Stats101 error of interpreting null effects–the absence of a reliable statistical effect of reading skill or interaction between reading skill and prior knowledge–as providing positive evidence that reading level didn’t matter, only background knowledge. That is not OK. The results are equally consistent with the conclusion that the groups didn’t differ in reading skill, or at least enough to have any measurable impact on the performance in this experiment.
The study didn’t yield a valid test of the impact of reading skill because of choices that were made in designing and conducting it. First, the differences between the good and poor readers were small because the authors excluded certain poor readers: “We selected low-ability subjects who scored above the 30th percentile in vocabulary to avoid word-recognition problems.” The weakest readers–the ones who really struggle with reading words–were excluded. Moreover, the simple tests that were used to assess reading skills have their own imprecision (margin of error). The net result is that the good and poor readers did not differ substantially in reading ability. The problem could have been avoided by choosing participants from a broader range of reading skills.
Second, there were only 16 participants in each condition. How many participants should there be in a study? Is 16 a large number or small? That depends on what’s being measured. Given the subtle, at best, differences between the good and poor readers, this number of participants does not yield enough data to detect differences between groups, should they exist. Statisticians would say the study lacked sufficient power to detect effects of reading skill. The study did have sufficient power to detect the effect of background knowledge because it is huge: readers who don’t know much about baseball will have great difficulty remembering what they read. Big effect: doesn’t take many observations to detect. Small effect: you’re gonna need a bigger sample.
Bottom line: though widely cited, the study is fatally flawed and the conclusions do not follow from the results. The study says nothing about the relative importance of background/prior knowledge versus reading skill. Don’t make the mistake of assuming that because a study is published the conclusions are correct.
I've never read the entire original baseball study, but I did just read the entire Seidenberg critique, and I feel compelled to write that I've never read anything written by Ms. Wexler (or anyone else who cited the baseball study) that claimed that background knowledge was more important than the ability to decode.
Both are necessary to comprehension, of course.
I'm a retired English teacher, so maybe things have gotten better since I left the profession, but I still talk to former colleagues from time to time, and I get the impression that the importance of subject knowledge is still underestimated in reading comprehension and critical thinking.
If I may recount an experience I had.
My last year of teaching I had moved to a new-to-me high school in a new-to-me district. The week before students returned we had one of those district wide professional development sessions specifically for English teachers, led by the district's Language Arts Coordinator. For most of the morning she impressed upon us the importance of creating a culture of readers. Kids would read if we surrounded them with books, especially books about topics they were interested in, if we gave them time to read what they wanted, and if we modeled being good readers ourselves.
The whole morning I sat there, thinking, Oh, crap, more of this B.S. Is she ever going to mention decoding and knowledge? Nope, she never did.
After her attempts to inspire us, we broke into groups by grade levels. I was sitting at a big table with a bunch of other teachers, trying to keep my mouth shut, when they started talking about the need to teach comprehension skills. In particular, they lamented at length students' inability to inference. I couldn't keep my mouth shut any longer.
"Hey, can I give you all a brief listening comprehension test?" I asked. "It's really short."
The fact that I was a new guy, and the oddness of my request surely had something to do with the looks I got, but a couple of heads nodded, so I went ahead. I used a passage, somewhat modified, that I stole from Ben Bergen's excellent book, 'Louder Than Words: The New Science of How We Make Meaning.' It goes like this:
Polar bears love seal meat. But seals are hard to catch, so the bears have come up with an ingenious way to hunt seals. They lower their chest to the ground, cover their nose with one paw, and use their back legs to push themselves forward and sneak up on the seals.
Then I asked the comprehension question: "Why do the bears cover their nose with their paw?"
Immediately, someone said, "So they blend in with the snow" and pretty much everyone else nodded their head in agreement.
"That's right," I said. "But did you notice this? I never mentioned where polar bears and seals live, I never mentioned snow or ice, and I never even mentioned a color."
I'm not exaggerating, all around the table mouths dropped open. I could see folk's eyes rolling up and looking off to the side, like they were replaying the passage in their heads.
I continued, "You filled in all that information yourself. And that's what inferencing really is: when you fill in the information the writer or speaker assumes you know and so refrains from explicitly mentioning. We are wired to make inferences; it's almost like you can't stop humans from doing it. The question is, do we have stored in long-term memory the information we need to make the correct inference?"
"And one more thing - think about how effortlessly and how instantly you retrieved that knowledge about bears and seals and snow from your long-term memory. That's why we tend to underestimate the role of knowledge in making inferences and comprehension. Once we have stuff firmly stored in long-term memory, using it can be almost automatic."
Yes, I know there's a difference between storage strength and retrieval strength, but I was keeping things simple.
I think they bought it; someone asked me to repeat that passage slowly and everyone at the table was typing away at their laptops as I did so.
I sincerely apologize if typing a comment this long is bad form. My point is, I don't know of anyone who claims that knowledge is more important than decoding is to reading comprehension, and just based on my anecdotal evidence, I'm not convinced all teachers (and administrators) understand the importance of subject knowledge.
I had forgotten that I'd left a comment on Mark's post back in 2020 -- and was about to more or less recreate it, so thanks for pointing that out. You've saved me some time. I'll just copy and paste it here as a response to Harriet's comment above:
This post seems to be attacking a straw man. The baseball study doesn’t argue that teaching word-reading skills is unnecessary and that background knowledge is all you need for reading. It’s a study of reading COMPREHENSION, not decoding. That’s why the researchers eliminated students who had scored below the 30th percentile on a standardized comprehension test. Maybe there are better ways of controlling for decoding ability, but the point is that they were trying to proceed on the assumption that the students being tested already HAD decoding skills.
Are there individuals out there who are using the baseball study to argue against the need to teach decoding skills, as this post seems to assume? I haven’t encountered any. That would, to my mind, be a clear distortion of its findings. The baseball study, and other studies that have found similar strong effects for background knowledge in regard to comprehension, shouldn’t be read to suggest that background knowledge is the ONLY component of reading ability. Yes, reading is complex. In addition to background knowledge and decoding ability, readers also need to be familiar with the conventions of written language, which is almost always more complex than spoken language. But the studies are useful illustrations of the key role of background knowledge — and the need to build it, not just activate it — which is all too often overlooked in the current “skills”-focused approach to comprehension instruction.
Both these fights about literacy--phonics and knowledge--just seem so ridiculous. I don't get it. They both seem like such basic obvious things we should be teaching. I mean, do the people on the other side of this debate ever stop and consider that they are literally fighting against giving kids knowledge? They are protesting that teachers should not, in fact, expect their students to know anything. On a purely logical level, how does that make sense?
From what I can tell, most people in “the knowledge camp” are quite familiar with research-based reading strategies and what this looks like in classrooms in the absence of a coherent, cumulative, knowledge-rich curriculum. As one baffled middle schooler I know recently said, “What is ELA even for, anyway?”
And I hear lots of “Yes, of course knowledge is important, BUT….” From what I can tell, there is still a huge anti-knowledge bias and plenty of misunderstandings out there among some researchers, teachers and teacher educators as evidenced by this denigration of knowledge as “mere facts”, paranoia about “drilling facts” and pushback against proponents of foregrounding content knowledge in schools. This content knowledge, along with teacher-directed active engagement with text and a volume of reading is where the focus should be, not, as another commenter here aptly put it, on“vaporware” strategies.
I'm not disagreeing that there are folks out there with wrongheaded opinions about strategies and knowledge; however, most of the published curriculum are now focused on central topics and themes. The idea of knowledge accumulation is embedded in lots of the published curriculum. I do however, see problems with curriculum that focus on knowledge without teaching strategies in a way that reflects the research, and also see problems with curriculums that still do skill of the week spirialization type instruction which also doesn't reflect the research base.
Regarding the knowledge proponents, I don't think Wexler has a working understanding of research based comprehension strategies and the gradual release - at least I've not read anything in her books or blogs that suggests she does. I have, however, read about the problems with 'skill of the week' over and over again. I honestly think she did a great service to the field to call out that type of instruction.
I don't think the Libens' really give comprehension strategies a fair run, or at least have a bias against them which leads them to strawman them at every turn. In their recently released book, the devoted next to no time talking about research based strategies.
Interestingly, in a recent podcast, Sue Pimentel said that main idea should be done everyday, but not in a route way. This confused me. If you are teaching a strategy for main idea, is that 'route'? If you aren't, what is your instruction? I can understand something like questioning the author being used for this purpose, but I'm a little confused by her stance at the moment.
People like Nell Duke, Dana Robertson, Tim Shanahan, and Dougherty Stahl talk and write about research based strategies and how they can be used to build knowledge. Shanahan puts it something like this - the focus of the lesson is the strategy, and the result of that focus is the knowledge which is gained.
I’ve heard the Libens refer to comprehension strategy instruction as “deadening”. So this is probably why they don’t spend too much time discussing it. Maybe you think this only refers to a strawman, but I have met this supposed strawman and he was very real. I think many others have too which is one reason Natalie Wexler’s book resonated with so many.
Making the strategy the focus of the lesson as Shanahan suggests is a sure way to make it deadening in my opinion. The knowledge should be the focus with any strategy or discussion technique in service of the knowledge and understanding to be gained from the reading.
I don't think we really disagree here; it's likely just semantics.
The strategy is the pathway to knowledge, which you agree with.
I say the strategy is the focus as it's the means to get to the content, which in my opinion is largely static and something to be acquired/integrated. We can quibble over which is the focus, but if you agree with what I've just said (which I suspect you do), I think we are on the same page.
Regarding the Libens', that's the exact definition of a strawman. They are presenting the worst version of strategies when they say those things....why not promote the research on the best version of strategies? They promote close reading extensively - which has almost no research based - but neglect strategy instruction because they don't like it....hardly 'science of reading'.
I think we agree that active engagement with text is needed and this can be fostered in different ways. I don’t think all the ways are equally beneficial though we don’t have much evidence one way or another because good comparative studies are practically non-existent. I actually do not agree that “the strategy is the pathway to knowledge” as a broad claim (although it may be for some students with some texts for some purposes).
I’m not sure how we can adjudicate if the LIbens’ portrayal of strategy instruction is a strawman or an accurate representation of typical practice. “Skill of the week” is egregious but there are other ways too IMO that strategies can be taught in “deadening” ways. We don’t have comprehensive descriptive data of what happens in classrooms across the country, so we have to triangulate and go on “anecdata”.
I think there is some actual theories behind this, but I think it might help illuminate the knowledge vs. a list of facts issue by thinking of knowledge as a mental model. We have a model of the world, into which new texts fit. If we don't have enough of a coherent model, a text that depends on a model won't make sense.
Geoffrey Hinton, a deep learning specialist who just won the Nobel Prize in Physics, has interesting and important things to say about how the human brain (and machine-based neural networks) UNDERSTAND language.
His work is mind-bending and thought-provoking with regard to what and how to educate students for the world they will inherit. See - https://www.youtube.com/watch?v=N1TEjTeQeg0
Today's Teaching Literacy Podcast (https://teachingliteracypodcast.com/ep57inferences-with-dr-marianne-rice/) is very informative, covering all the usual suspects including, most broadly, knowledge-building and strategy instruction. At the end Rice discusses the importance of establishing a high-utility routine that can be applied to ALL texts. She recommends 1) text structure analysis, (2) getting the "gist" (main idea instruction including summary writing), and (3) inference-making (two categories: knowledge-based and text-based).
All the knowledge based curriculums I have reviewed, inter-weave comprehension focused skills within their reading, writing and discussions. I am stymied by the ongoing skills vs. knowledge content discussions. I simply cannot imagine teaching knowledge without integrating vocabulary/morphology instruction within a written product wrapped in rich discussion. At the end of the day students will remember the quirky story or personal connection teachers have to the knowledge they present. I refuse to make comprehension instruction drudgery, either through isolated skills instruction or lecture/ factoid knowledge memorization ( which I’m not sure is really in the students long term memory anyway). Why can’t we get back to sharing our excitement of learning new things?
Lots of them don't actually teach strategies. They ask kids questions and get them to do teacher directed activities that 'might' use the same language as strategies... Eg summarise or infer, but aren't actually teaching kids to be independent in the use of these strategies.
The whole idea behind strategies is that they are conscious actions kids take to better understand a text. I've heard sue Pimentel and others say that strategies are in the background and that it's hard to know they are even there. This isn't possible if strategies are defined as conscious actions kids are taking to better understand the text.
This is an important point. Strategies are by definition conscious actions and if the claim is that they become "unconscious" then well, they are no longer strategies. I have heard lots of advocates of teaching strategies miss this point. As conscious actions, they inevitable carry a cognitive load, as Natalie notes in this post when she discusses needing to look up unfamiliar words.
Also, it's always worth considering Margaret McKeown's 2009 study comparing strategy instruction (summarizing, predicting, drawing inferences, question generation, and comprehension monitoring) with content-oriented discussion that engaged students in attending to the ideas and building a mental model of the text with *no direction* to consider specific mental processes as in the strategy approach. Outcomes favored the content-oriented group. It's beyond me why the unproductive arguments about reading comprehension strategy instruction seem to go on and on without anyone trying to replicate or expand on this study.
McKeown, M. G., Beck, I. L., & Blake, R. G. K. (2009). Rethinking reading comprehension instruction: A comparison of instruction for strategies and content approaches. Reading Research Quarterly, 44(3), 218–253.
Thanks for reminding us of this study. I will definitely reread, especially to remind myself what "sentence verification technique" refers to. From the abstract:
No differences were seen on one measure of lesson-text comprehension, the sentence verification technique. However, for narrative recall and expository learning probes, content students outperformed strategies students.
While I think this study is interesting, it doesn't use standardised measures and is only one study.
Moreover, I'd argue that the content approach in the study 'questioning the author' is a strategy or routine in itself....the idea behind it is that you are changing how kids approach a text...so as far as strategies and QtA, I believe they both teach the same thing - active and engaged processing of text.
The instruction in a lot of these knowledge building curriculum is not similar to Questioning the Author. As I mentioned previously, it is teacher directed and asks kids to do tasks and answer questions.
In summary, the QtA study can be used as evidence that QtA could replace strategy instruction (although it needs much more support to firm up that claim), but it can't be used to support the idea of knowledge rich curriculum.
I've heard this criticism of knowledge-building curricula before--i.e., that they're "teacher directed" and just "ask kids to do tasks and answer questions," and I have to wonder what these critics are basing their criticism on.
Of course, no curriculum is perfect, and any good curriculum can be implemented in a way that doesn't work. But I've spent dozens (maybe hundreds) of hours observing several of these curricula in action in classrooms, and I've spoken with many teachers who have implemented them.
What I've seen and heard is actually the opposite of what these critics are saying. Yes, knowledge-building curricula are "teacher directed," but I don't see that as a bad thing. Lots of evidence indicates that teacher-directed instruction (along with lots of student interaction) works better than student-directed instruction.
And the idea that kids aren't learning to independently make inferences, etc., through these curricula has no basis in reality, as far as I can see. On the contrary, teachers tell me that kids often start spontaneously making connections and inferences, etc. -- definitely more than they did with a curriculum focused on comprehension skills and strategies. I've seen that happen myself.
If there are studies out there indicating that knowledge-building curricula prevent kids from independently engaging with and processing text, I'd like to know about them.
You missed my point. I'm saying that doing teacher directed activities and answering teacher questions isn't comprehension strategy instruction. I'm not saying that's bad and doesn't have a place, but it isn't comprehension strategy instruction.
I feel like sometimes you hedge your bets by saying these curriculum include comprehension strategy instruction but then also say that the activities are teacher directed and guide the students through understanding the text
(which in your eyes is superior to comprehension strategy instruction). So which one is it? Do these curriculum include comprehension strategy instruction or do they do something, that in your eyes, is better than comprehension strategy instruction?
Lastly, why not teach research comprehension strategies with the knowledge rich curriculum you advocate for? Apply the research based strategies (text structure Strategy, read stop write, collaborative strategic reasoning ect) and processes with the sequenced knowledge building curriculum. Do you have an objection to this idea? If so, on what empirical grounds?
QtA is in fact very teacher-directed, with the teacher carefully choosing pause points and formulating queries and follow ups. Eventually students are scaffolded and helped to become increasingly independent as they read and make sense of tests. I agree that any fostering of active engagement with text is bound to be helpful and this is why we tend to see evidence of benefits of strategy instruction.. You may not see a distinction, but Margaret McKeown would disagree with you that this approach is just another variety of teaching strategies. It eliminates any kind of “strategic layer” or focus on a routine or metacognitive awareness. The focus is squarely on the text and on understanding it. It is just one study, but it is proof of concept that all active engagement with text may not be equally effective. I wish researchers would do more of these kinds of studies.
You’re right,this study is not really relevant to arguments for knowledge rich curriculum. The nature of language itself stands as the strongest argument for a knowledge rich curriculum (not to mention the inherent value of knowledge).
I didn't mean to say strategies and QtA were the same thing, but rather that they have they achieve the same goal: active processing of text. . . I believe McKeown says this in her book. I'm not sure that teacher directed questioning and text dependent/close reading method of teaching achieves the same goals. . .
I wasn't sure whether ChatGPT subscribes to your substack, so I asked it to weigh in on the impact of teaching Tier 2 words (general academic vocabulary) and Tier 3 words (content-specific vocabulary). Amidst the brouhaha over knowledge-building vs. strategy instruction, what we don't want to lose cite of is the importance of teaching Tier 2 words within the context of our chosen curriculum.
From ChatGPT:
Comparison of Impact:
Tier 2 words tend to have a wider-reaching effect on reading comprehension since they are used across many domains and academic disciplines. A strong understanding of Tier 2 words helps readers tackle a broad range of texts.
Tier 3 words, while essential for understanding specific subjects or disciplines, have a more narrow and domain-specific influence. They are important for content-specific comprehension but don't affect the overall ability to understand non-specialized academic texts as much.
Tier 2 words are important and the Bringing Words to Life approach by Beck, et al is valuable in my opinion. But E.D. Hirsch Jr. disagrees with Chat GPT about Tier 3 words. I'm with him :)
Yes, knowledge of a topic helps facilitate reading comprehension. We get it. I think that message has been drummed home hard over the past few years and is well understood. More articles on this is topic is continuing to beat a dead horse.
Now let's give research based comprehension strategies their due without using the 'skill of the week' strawman.
Let's talk about the studies that explicitly teach summarizing, main idea, text structure, and comprehension monitoring. Let's talk about how to embed this in a way that reflects the research and also facilitates the building of knowledge.
Look at the work of Joan Sedita, Sharon Vaughan, and Wijekumar. This is what research based comprehension strategy instruction looks like. It builds knowledge and use of strategies and schemas. Strategies are taught and modelled briefly before being used as tools to mine complex and worthwhile texts.
Talk to (or read the work) of other researchers who know the research base on comprehension strategies and who have taught in the classroom eg. Dana Robertson, John Z. Strong, Sharon Vaughan, Dougherty-Stahl, and others mentioned above.
In short, give research based comprehension strategies their due like you have knowledge.
Sharon Vaughn is one of my go-to researchers. She is part of the team who wrote the IES Practice Guide for grades 4-9 (https://ies.ed.gov/ncee/wwc/practiceguide/29), where they provide strong evidence for the following:
Strong Evidence
Routinely use a set of comprehension-building practices to help students make sense of the text
Part 3A. Build students’ world and word knowledge so they can make sense of the text
Part 3B. Consistently provide students with opportunities to ask and answer questions to better understand the text they read
Part 3C. Teach students a routine for determining the gist of a short section of text
Part 3D. Teach students to monitor their comprehension as they read
My version of "get the gist" is "paragraph shrinking," which I explain in my instructional guide to reading, From Sound to Summary: Braiding the Reading Rope to Make Words Make Sense. I also spend teaching time on something Natalie emphasizes: syntax. I refer to Alex Quigley's article, "Crafting Great Sentences," (https://alexquigley.co.uk/crafting-great-sentences/), where he provides good examples for sentence combining, expanding, shrinking, and signposting--which not only improve student writing but also student comprehension.
Great essay. I always feel the point for me at least of the Baseball Study is that if we had standardized curricular topics by grade, standardized tests could be focused on those topics and schools could dedicate time to teaching deeply on those topics instead of focusing so heavily on strategy. One year my son was abroad and being taught by an Irish teacher and was assigned a novel on hurling and it was a brilliant personal Baseball Study for both him and I. It was clear that a simple 4th grade level book was impossible to understand without going back and learning the content first.
I would rather have students drilling facts that useful than strategies. Knowing the human body, knowing the year a war started, knowing the parts of tree…is knowledge a child can take with them. The strategies that consume so much time are like vaporware. Of course anything can be done poorly but it is also possible that a well structured curriculum could provide a strong basis of communal knowledge. If we are going to have standardized assessments, and we are; then it is better they are testing knowledge that is grade appropriate and worth having versus random passages with no connection to what students are learning in class.
I guess it's really a matter of opinion, but I can see serious problems with drilling facts to 'improve' reading comprehension. I could easily see this leading to less actual reading instruction and a reduction of research based comprehension strategies (not like the ones you were referring to).
Wouldn't this just result in schools drilling facts and details about assessed topics to increase reading comprehension scores? I know this isn't the intention behind such assessment schemes, but I find it hard to believe that schools wouldn't end up doing that. . . It'd kind achieve the same lethal mutation that occured with comprehension assessments that reported on question types like they were valid constructs.
And this thing is, if we take the evidence from studies such as the baseball study at face value, cramming facts and knowledge about certain topics that are tested will improve reading comprehension on those assessments....but won't likely help much on topics not assessed.
Doesn't seem like the most well thought out assessment solution if you ask me.
As the father of three children (all now in college), I find it shocking that this must even be argued. People will be more engaged with a text on a topic they understand, using words they understand. Those who claim that "strategies" are paramount seem to regard humans as mere computers processesing information. What a shame that any such persons are involved in education.
Those that claim strategies are important general don't claim topical knowledge isn't. That's a big misconception.
Strategies are what helps kids build knowledge when reading deeply about a topic.
Great article. I find that literacy teachers forget what is fundamental about words: that they are symbols that represent reality. The skills to decode symbols are meaningless without knowledge of the underlying reality they point to.
I observe the phenomenon mentioned here in my own reading. If I research a topic online that has specialized terminology (scientific, for instance) there's a tipping point at which there are too many unfamiliar terms for me to really comprehend a passage. My "working memory" becomes overwhelmed. To understand a passage and make inferences about words you don't know there have to be a sufficient amount of words you do understand.
And besides the question of making inferences, if it is too hard to comprehend the passage you aren't going to experiencing the reward of gaining comprehension so your motivation decreases.
Might I suggest researchers start looking at how countries with high literacy and comprehensive skill rates educate their children. Seems like the practical thing to do.
Some researchers have done that, although trying to transplant practices that work in one country to another country is tricky. There can be huge differences in context and culture. Generally, however, the research has found that countries with a clearly defined national curriculum and tests aligned to the content of that curriculum do better in international comparisons.
Education in the U.S. is highly localized, so there's no possibility of a national curriculum here. No state even has a mandatory curriculum, although legally they could. But there's a lot that can be done at the state level to encourage adoption of content-rich, knowledge-building curricula. And states could ground the passages on reading tests in their content standards in science and social studies.
Oh, I totally understand that but looking at other successful methods could generate useful ideas to incorporate into discussions. It sounds like people are talking past each other instead of to each other. Proof of a working concept could help with that.
That sort of bothers me that we don't have a national curriculum. Children in red states are being short changed because of it. For instance, when I was growing up in Ohio I got a first class public education. Fast forward 40 years and education isn't that great in Ohio anymore.
That sounds like a great idea! Having even a little background knowledge in a subject is very helpful for comprehension. I used to read complex medical studies just for fun, lol. Even though I'm not a doctor I understood quite a lot with the little background knowledge I do have.
Thank you for posting another thought provoking article.
Here's Mark Seidenberg's two cents on the baseball study:
https://seidenbergreading.net/2020/09/10/some-context-on-context/
Folks, the study says exactly nothing about the relative importance of background knowledge and reading skills. The authors wanted to compare good and poor readers but didn’t succeed in doing so. There are no effects of reader group in this study. The procedures used to select participants did not yield groups that differed in reading skill. The study therefore does not show anything about the effect of reading skill or the interaction of reading skill and prior knowledge. The experiment would have yielded the same results if the texts had been read aloud to the participants–if there were no reading involved at all.
The authors made the Stats101 error of interpreting null effects–the absence of a reliable statistical effect of reading skill or interaction between reading skill and prior knowledge–as providing positive evidence that reading level didn’t matter, only background knowledge. That is not OK. The results are equally consistent with the conclusion that the groups didn’t differ in reading skill, or at least enough to have any measurable impact on the performance in this experiment.
The study didn’t yield a valid test of the impact of reading skill because of choices that were made in designing and conducting it. First, the differences between the good and poor readers were small because the authors excluded certain poor readers: “We selected low-ability subjects who scored above the 30th percentile in vocabulary to avoid word-recognition problems.” The weakest readers–the ones who really struggle with reading words–were excluded. Moreover, the simple tests that were used to assess reading skills have their own imprecision (margin of error). The net result is that the good and poor readers did not differ substantially in reading ability. The problem could have been avoided by choosing participants from a broader range of reading skills.
Second, there were only 16 participants in each condition. How many participants should there be in a study? Is 16 a large number or small? That depends on what’s being measured. Given the subtle, at best, differences between the good and poor readers, this number of participants does not yield enough data to detect differences between groups, should they exist. Statisticians would say the study lacked sufficient power to detect effects of reading skill. The study did have sufficient power to detect the effect of background knowledge because it is huge: readers who don’t know much about baseball will have great difficulty remembering what they read. Big effect: doesn’t take many observations to detect. Small effect: you’re gonna need a bigger sample.
Bottom line: though widely cited, the study is fatally flawed and the conclusions do not follow from the results. The study says nothing about the relative importance of background/prior knowledge versus reading skill. Don’t make the mistake of assuming that because a study is published the conclusions are correct.
I've never read the entire original baseball study, but I did just read the entire Seidenberg critique, and I feel compelled to write that I've never read anything written by Ms. Wexler (or anyone else who cited the baseball study) that claimed that background knowledge was more important than the ability to decode.
Both are necessary to comprehension, of course.
I'm a retired English teacher, so maybe things have gotten better since I left the profession, but I still talk to former colleagues from time to time, and I get the impression that the importance of subject knowledge is still underestimated in reading comprehension and critical thinking.
If I may recount an experience I had.
My last year of teaching I had moved to a new-to-me high school in a new-to-me district. The week before students returned we had one of those district wide professional development sessions specifically for English teachers, led by the district's Language Arts Coordinator. For most of the morning she impressed upon us the importance of creating a culture of readers. Kids would read if we surrounded them with books, especially books about topics they were interested in, if we gave them time to read what they wanted, and if we modeled being good readers ourselves.
The whole morning I sat there, thinking, Oh, crap, more of this B.S. Is she ever going to mention decoding and knowledge? Nope, she never did.
After her attempts to inspire us, we broke into groups by grade levels. I was sitting at a big table with a bunch of other teachers, trying to keep my mouth shut, when they started talking about the need to teach comprehension skills. In particular, they lamented at length students' inability to inference. I couldn't keep my mouth shut any longer.
"Hey, can I give you all a brief listening comprehension test?" I asked. "It's really short."
The fact that I was a new guy, and the oddness of my request surely had something to do with the looks I got, but a couple of heads nodded, so I went ahead. I used a passage, somewhat modified, that I stole from Ben Bergen's excellent book, 'Louder Than Words: The New Science of How We Make Meaning.' It goes like this:
Polar bears love seal meat. But seals are hard to catch, so the bears have come up with an ingenious way to hunt seals. They lower their chest to the ground, cover their nose with one paw, and use their back legs to push themselves forward and sneak up on the seals.
Then I asked the comprehension question: "Why do the bears cover their nose with their paw?"
Immediately, someone said, "So they blend in with the snow" and pretty much everyone else nodded their head in agreement.
"That's right," I said. "But did you notice this? I never mentioned where polar bears and seals live, I never mentioned snow or ice, and I never even mentioned a color."
I'm not exaggerating, all around the table mouths dropped open. I could see folk's eyes rolling up and looking off to the side, like they were replaying the passage in their heads.
I continued, "You filled in all that information yourself. And that's what inferencing really is: when you fill in the information the writer or speaker assumes you know and so refrains from explicitly mentioning. We are wired to make inferences; it's almost like you can't stop humans from doing it. The question is, do we have stored in long-term memory the information we need to make the correct inference?"
"And one more thing - think about how effortlessly and how instantly you retrieved that knowledge about bears and seals and snow from your long-term memory. That's why we tend to underestimate the role of knowledge in making inferences and comprehension. Once we have stuff firmly stored in long-term memory, using it can be almost automatic."
Yes, I know there's a difference between storage strength and retrieval strength, but I was keeping things simple.
I think they bought it; someone asked me to repeat that passage slowly and everyone at the table was typing away at their laptops as I did so.
I sincerely apologize if typing a comment this long is bad form. My point is, I don't know of anyone who claims that knowledge is more important than decoding is to reading comprehension, and just based on my anecdotal evidence, I'm not convinced all teachers (and administrators) understand the importance of subject knowledge.
Your "experiment" was brilliant! You managed to cut through the background noise and make an important point. I love the KIS method. Keep it simple.
Thank you.
You're welcome.
Shoot! I forgot to add that the story about the way polar bears hunt seals is apocryphal.
Thanks for sharing that critique. Did you notice that Ms Wexler is in the comments section of that article, responding in 2020?
I had forgotten that I'd left a comment on Mark's post back in 2020 -- and was about to more or less recreate it, so thanks for pointing that out. You've saved me some time. I'll just copy and paste it here as a response to Harriet's comment above:
This post seems to be attacking a straw man. The baseball study doesn’t argue that teaching word-reading skills is unnecessary and that background knowledge is all you need for reading. It’s a study of reading COMPREHENSION, not decoding. That’s why the researchers eliminated students who had scored below the 30th percentile on a standardized comprehension test. Maybe there are better ways of controlling for decoding ability, but the point is that they were trying to proceed on the assumption that the students being tested already HAD decoding skills.
Are there individuals out there who are using the baseball study to argue against the need to teach decoding skills, as this post seems to assume? I haven’t encountered any. That would, to my mind, be a clear distortion of its findings. The baseball study, and other studies that have found similar strong effects for background knowledge in regard to comprehension, shouldn’t be read to suggest that background knowledge is the ONLY component of reading ability. Yes, reading is complex. In addition to background knowledge and decoding ability, readers also need to be familiar with the conventions of written language, which is almost always more complex than spoken language. But the studies are useful illustrations of the key role of background knowledge — and the need to build it, not just activate it — which is all too often overlooked in the current “skills”-focused approach to comprehension instruction.
Both these fights about literacy--phonics and knowledge--just seem so ridiculous. I don't get it. They both seem like such basic obvious things we should be teaching. I mean, do the people on the other side of this debate ever stop and consider that they are literally fighting against giving kids knowledge? They are protesting that teachers should not, in fact, expect their students to know anything. On a purely logical level, how does that make sense?
No, they don't. I think the issue largely resides with the knowledge camp misunderstanding what research based comprehension strategies actually are.
From what I can tell, most people in “the knowledge camp” are quite familiar with research-based reading strategies and what this looks like in classrooms in the absence of a coherent, cumulative, knowledge-rich curriculum. As one baffled middle schooler I know recently said, “What is ELA even for, anyway?”
And I hear lots of “Yes, of course knowledge is important, BUT….” From what I can tell, there is still a huge anti-knowledge bias and plenty of misunderstandings out there among some researchers, teachers and teacher educators as evidenced by this denigration of knowledge as “mere facts”, paranoia about “drilling facts” and pushback against proponents of foregrounding content knowledge in schools. This content knowledge, along with teacher-directed active engagement with text and a volume of reading is where the focus should be, not, as another commenter here aptly put it, on“vaporware” strategies.
I'm not disagreeing that there are folks out there with wrongheaded opinions about strategies and knowledge; however, most of the published curriculum are now focused on central topics and themes. The idea of knowledge accumulation is embedded in lots of the published curriculum. I do however, see problems with curriculum that focus on knowledge without teaching strategies in a way that reflects the research, and also see problems with curriculums that still do skill of the week spirialization type instruction which also doesn't reflect the research base.
Regarding the knowledge proponents, I don't think Wexler has a working understanding of research based comprehension strategies and the gradual release - at least I've not read anything in her books or blogs that suggests she does. I have, however, read about the problems with 'skill of the week' over and over again. I honestly think she did a great service to the field to call out that type of instruction.
I don't think the Libens' really give comprehension strategies a fair run, or at least have a bias against them which leads them to strawman them at every turn. In their recently released book, the devoted next to no time talking about research based strategies.
Interestingly, in a recent podcast, Sue Pimentel said that main idea should be done everyday, but not in a route way. This confused me. If you are teaching a strategy for main idea, is that 'route'? If you aren't, what is your instruction? I can understand something like questioning the author being used for this purpose, but I'm a little confused by her stance at the moment.
People like Nell Duke, Dana Robertson, Tim Shanahan, and Dougherty Stahl talk and write about research based strategies and how they can be used to build knowledge. Shanahan puts it something like this - the focus of the lesson is the strategy, and the result of that focus is the knowledge which is gained.
I’ve heard the Libens refer to comprehension strategy instruction as “deadening”. So this is probably why they don’t spend too much time discussing it. Maybe you think this only refers to a strawman, but I have met this supposed strawman and he was very real. I think many others have too which is one reason Natalie Wexler’s book resonated with so many.
Making the strategy the focus of the lesson as Shanahan suggests is a sure way to make it deadening in my opinion. The knowledge should be the focus with any strategy or discussion technique in service of the knowledge and understanding to be gained from the reading.
I don't think we really disagree here; it's likely just semantics.
The strategy is the pathway to knowledge, which you agree with.
I say the strategy is the focus as it's the means to get to the content, which in my opinion is largely static and something to be acquired/integrated. We can quibble over which is the focus, but if you agree with what I've just said (which I suspect you do), I think we are on the same page.
Regarding the Libens', that's the exact definition of a strawman. They are presenting the worst version of strategies when they say those things....why not promote the research on the best version of strategies? They promote close reading extensively - which has almost no research based - but neglect strategy instruction because they don't like it....hardly 'science of reading'.
I think we agree that active engagement with text is needed and this can be fostered in different ways. I don’t think all the ways are equally beneficial though we don’t have much evidence one way or another because good comparative studies are practically non-existent. I actually do not agree that “the strategy is the pathway to knowledge” as a broad claim (although it may be for some students with some texts for some purposes).
I’m not sure how we can adjudicate if the LIbens’ portrayal of strategy instruction is a strawman or an accurate representation of typical practice. “Skill of the week” is egregious but there are other ways too IMO that strategies can be taught in “deadening” ways. We don’t have comprehensive descriptive data of what happens in classrooms across the country, so we have to triangulate and go on “anecdata”.
I think there is some actual theories behind this, but I think it might help illuminate the knowledge vs. a list of facts issue by thinking of knowledge as a mental model. We have a model of the world, into which new texts fit. If we don't have enough of a coherent model, a text that depends on a model won't make sense.
Geoffrey Hinton, a deep learning specialist who just won the Nobel Prize in Physics, has interesting and important things to say about how the human brain (and machine-based neural networks) UNDERSTAND language.
His work is mind-bending and thought-provoking with regard to what and how to educate students for the world they will inherit. See - https://www.youtube.com/watch?v=N1TEjTeQeg0
Today's Teaching Literacy Podcast (https://teachingliteracypodcast.com/ep57inferences-with-dr-marianne-rice/) is very informative, covering all the usual suspects including, most broadly, knowledge-building and strategy instruction. At the end Rice discusses the importance of establishing a high-utility routine that can be applied to ALL texts. She recommends 1) text structure analysis, (2) getting the "gist" (main idea instruction including summary writing), and (3) inference-making (two categories: knowledge-based and text-based).
All the knowledge based curriculums I have reviewed, inter-weave comprehension focused skills within their reading, writing and discussions. I am stymied by the ongoing skills vs. knowledge content discussions. I simply cannot imagine teaching knowledge without integrating vocabulary/morphology instruction within a written product wrapped in rich discussion. At the end of the day students will remember the quirky story or personal connection teachers have to the knowledge they present. I refuse to make comprehension instruction drudgery, either through isolated skills instruction or lecture/ factoid knowledge memorization ( which I’m not sure is really in the students long term memory anyway). Why can’t we get back to sharing our excitement of learning new things?
Lots of them don't actually teach strategies. They ask kids questions and get them to do teacher directed activities that 'might' use the same language as strategies... Eg summarise or infer, but aren't actually teaching kids to be independent in the use of these strategies.
The whole idea behind strategies is that they are conscious actions kids take to better understand a text. I've heard sue Pimentel and others say that strategies are in the background and that it's hard to know they are even there. This isn't possible if strategies are defined as conscious actions kids are taking to better understand the text.
This is an important point. Strategies are by definition conscious actions and if the claim is that they become "unconscious" then well, they are no longer strategies. I have heard lots of advocates of teaching strategies miss this point. As conscious actions, they inevitable carry a cognitive load, as Natalie notes in this post when she discusses needing to look up unfamiliar words.
Also, it's always worth considering Margaret McKeown's 2009 study comparing strategy instruction (summarizing, predicting, drawing inferences, question generation, and comprehension monitoring) with content-oriented discussion that engaged students in attending to the ideas and building a mental model of the text with *no direction* to consider specific mental processes as in the strategy approach. Outcomes favored the content-oriented group. It's beyond me why the unproductive arguments about reading comprehension strategy instruction seem to go on and on without anyone trying to replicate or expand on this study.
McKeown, M. G., Beck, I. L., & Blake, R. G. K. (2009). Rethinking reading comprehension instruction: A comparison of instruction for strategies and content approaches. Reading Research Quarterly, 44(3), 218–253.
Thanks for reminding us of this study. I will definitely reread, especially to remind myself what "sentence verification technique" refers to. From the abstract:
No differences were seen on one measure of lesson-text comprehension, the sentence verification technique. However, for narrative recall and expository learning probes, content students outperformed strategies students.
While I think this study is interesting, it doesn't use standardised measures and is only one study.
Moreover, I'd argue that the content approach in the study 'questioning the author' is a strategy or routine in itself....the idea behind it is that you are changing how kids approach a text...so as far as strategies and QtA, I believe they both teach the same thing - active and engaged processing of text.
The instruction in a lot of these knowledge building curriculum is not similar to Questioning the Author. As I mentioned previously, it is teacher directed and asks kids to do tasks and answer questions.
In summary, the QtA study can be used as evidence that QtA could replace strategy instruction (although it needs much more support to firm up that claim), but it can't be used to support the idea of knowledge rich curriculum.
I've heard this criticism of knowledge-building curricula before--i.e., that they're "teacher directed" and just "ask kids to do tasks and answer questions," and I have to wonder what these critics are basing their criticism on.
Of course, no curriculum is perfect, and any good curriculum can be implemented in a way that doesn't work. But I've spent dozens (maybe hundreds) of hours observing several of these curricula in action in classrooms, and I've spoken with many teachers who have implemented them.
What I've seen and heard is actually the opposite of what these critics are saying. Yes, knowledge-building curricula are "teacher directed," but I don't see that as a bad thing. Lots of evidence indicates that teacher-directed instruction (along with lots of student interaction) works better than student-directed instruction.
And the idea that kids aren't learning to independently make inferences, etc., through these curricula has no basis in reality, as far as I can see. On the contrary, teachers tell me that kids often start spontaneously making connections and inferences, etc. -- definitely more than they did with a curriculum focused on comprehension skills and strategies. I've seen that happen myself.
If there are studies out there indicating that knowledge-building curricula prevent kids from independently engaging with and processing text, I'd like to know about them.
You missed my point. I'm saying that doing teacher directed activities and answering teacher questions isn't comprehension strategy instruction. I'm not saying that's bad and doesn't have a place, but it isn't comprehension strategy instruction.
I feel like sometimes you hedge your bets by saying these curriculum include comprehension strategy instruction but then also say that the activities are teacher directed and guide the students through understanding the text
(which in your eyes is superior to comprehension strategy instruction). So which one is it? Do these curriculum include comprehension strategy instruction or do they do something, that in your eyes, is better than comprehension strategy instruction?
Lastly, why not teach research comprehension strategies with the knowledge rich curriculum you advocate for? Apply the research based strategies (text structure Strategy, read stop write, collaborative strategic reasoning ect) and processes with the sequenced knowledge building curriculum. Do you have an objection to this idea? If so, on what empirical grounds?
QtA is in fact very teacher-directed, with the teacher carefully choosing pause points and formulating queries and follow ups. Eventually students are scaffolded and helped to become increasingly independent as they read and make sense of tests. I agree that any fostering of active engagement with text is bound to be helpful and this is why we tend to see evidence of benefits of strategy instruction.. You may not see a distinction, but Margaret McKeown would disagree with you that this approach is just another variety of teaching strategies. It eliminates any kind of “strategic layer” or focus on a routine or metacognitive awareness. The focus is squarely on the text and on understanding it. It is just one study, but it is proof of concept that all active engagement with text may not be equally effective. I wish researchers would do more of these kinds of studies.
You’re right,this study is not really relevant to arguments for knowledge rich curriculum. The nature of language itself stands as the strongest argument for a knowledge rich curriculum (not to mention the inherent value of knowledge).
I didn't mean to say strategies and QtA were the same thing, but rather that they have they achieve the same goal: active processing of text. . . I believe McKeown says this in her book. I'm not sure that teacher directed questioning and text dependent/close reading method of teaching achieves the same goals. . .
I wasn't sure whether ChatGPT subscribes to your substack, so I asked it to weigh in on the impact of teaching Tier 2 words (general academic vocabulary) and Tier 3 words (content-specific vocabulary). Amidst the brouhaha over knowledge-building vs. strategy instruction, what we don't want to lose cite of is the importance of teaching Tier 2 words within the context of our chosen curriculum.
From ChatGPT:
Comparison of Impact:
Tier 2 words tend to have a wider-reaching effect on reading comprehension since they are used across many domains and academic disciplines. A strong understanding of Tier 2 words helps readers tackle a broad range of texts.
Tier 3 words, while essential for understanding specific subjects or disciplines, have a more narrow and domain-specific influence. They are important for content-specific comprehension but don't affect the overall ability to understand non-specialized academic texts as much.
That's lose SIGHT of! I was thinking that it would be nice to cite some sources on vocabulary instruction related to comprehension.
Tier 2 words are important and the Bringing Words to Life approach by Beck, et al is valuable in my opinion. But E.D. Hirsch Jr. disagrees with Chat GPT about Tier 3 words. I'm with him :)
In my opinion, this is one of the best articles on vocabulary and reading comprehension. It's well worth your time. https://www.aft.org/sites/default/files/Adams.pdf
Excellent article, Miriam. Thank you!
Yes, knowledge of a topic helps facilitate reading comprehension. We get it. I think that message has been drummed home hard over the past few years and is well understood. More articles on this is topic is continuing to beat a dead horse.
Now let's give research based comprehension strategies their due without using the 'skill of the week' strawman.
Let's talk about the studies that explicitly teach summarizing, main idea, text structure, and comprehension monitoring. Let's talk about how to embed this in a way that reflects the research and also facilitates the building of knowledge.
Look at the work of Joan Sedita, Sharon Vaughan, and Wijekumar. This is what research based comprehension strategy instruction looks like. It builds knowledge and use of strategies and schemas. Strategies are taught and modelled briefly before being used as tools to mine complex and worthwhile texts.
Talk to (or read the work) of other researchers who know the research base on comprehension strategies and who have taught in the classroom eg. Dana Robertson, John Z. Strong, Sharon Vaughan, Dougherty-Stahl, and others mentioned above.
In short, give research based comprehension strategies their due like you have knowledge.
Sharon Vaughn is one of my go-to researchers. She is part of the team who wrote the IES Practice Guide for grades 4-9 (https://ies.ed.gov/ncee/wwc/practiceguide/29), where they provide strong evidence for the following:
Strong Evidence
Routinely use a set of comprehension-building practices to help students make sense of the text
Part 3A. Build students’ world and word knowledge so they can make sense of the text
Part 3B. Consistently provide students with opportunities to ask and answer questions to better understand the text they read
Part 3C. Teach students a routine for determining the gist of a short section of text
Part 3D. Teach students to monitor their comprehension as they read
My version of "get the gist" is "paragraph shrinking," which I explain in my instructional guide to reading, From Sound to Summary: Braiding the Reading Rope to Make Words Make Sense. I also spend teaching time on something Natalie emphasizes: syntax. I refer to Alex Quigley's article, "Crafting Great Sentences," (https://alexquigley.co.uk/crafting-great-sentences/), where he provides good examples for sentence combining, expanding, shrinking, and signposting--which not only improve student writing but also student comprehension.
Great essay. I always feel the point for me at least of the Baseball Study is that if we had standardized curricular topics by grade, standardized tests could be focused on those topics and schools could dedicate time to teaching deeply on those topics instead of focusing so heavily on strategy. One year my son was abroad and being taught by an Irish teacher and was assigned a novel on hurling and it was a brilliant personal Baseball Study for both him and I. It was clear that a simple 4th grade level book was impossible to understand without going back and learning the content first.
I would rather have students drilling facts that useful than strategies. Knowing the human body, knowing the year a war started, knowing the parts of tree…is knowledge a child can take with them. The strategies that consume so much time are like vaporware. Of course anything can be done poorly but it is also possible that a well structured curriculum could provide a strong basis of communal knowledge. If we are going to have standardized assessments, and we are; then it is better they are testing knowledge that is grade appropriate and worth having versus random passages with no connection to what students are learning in class.
I guess it's really a matter of opinion, but I can see serious problems with drilling facts to 'improve' reading comprehension. I could easily see this leading to less actual reading instruction and a reduction of research based comprehension strategies (not like the ones you were referring to).
Wouldn't this just result in schools drilling facts and details about assessed topics to increase reading comprehension scores? I know this isn't the intention behind such assessment schemes, but I find it hard to believe that schools wouldn't end up doing that. . . It'd kind achieve the same lethal mutation that occured with comprehension assessments that reported on question types like they were valid constructs.
And this thing is, if we take the evidence from studies such as the baseball study at face value, cramming facts and knowledge about certain topics that are tested will improve reading comprehension on those assessments....but won't likely help much on topics not assessed.
Doesn't seem like the most well thought out assessment solution if you ask me.