Harvard Professor Martin West on This Week’s Harrowing NAEP Results

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

Thursday’s release of the first COVID-era scores from the National Assessment of Educational Progress validated the public’s worst fears about pandemic learning loss.

The results of the benchmark federal exam, referred to as the “Nation’s Report Card,” revealed startling declines in student performance, including the first-ever drop in long-term math scores. Nine-year-olds, who have made steady progress since the test was first administered in the early 1970s, saw roughly twenty years of measured growth evaporate between 2020 and 2022. 

Using the results of state standardized tests, as well as private assessments like the MAP or iReady exams, a growing cadre of academics have offered evidence of K-12 learning deficits produced COVID’s disruption to school operations — including signs that Democratic-leaning states and districts, which were more likely to close schools longer, saw less instruction and steeper hits to achievement. But NAEP provides the first nationally representative data confirming those suspicions and charting the diverging effects on distinct student groups.

While average math and English scores fell for virtually all students, historically disadvantaged children — among them African Americans, Hispanics, the poor, and academically struggling students — generally saw larger drops, widening the gaps with their higher-scoring peers. 

Martin West is the academic dean and Henry Less Shattuck professor of education at the Harvard Graduate School of Education. As a member of the National Assessment Governing Board, an independent, nonpartisan body that sets policy for NAEP, he also has a unique perspective on the test and the data it generates.

In an email exchange Thursday with The 74’s Kevin Mahnken, West spoke about the results, the possibility of growing educational inequity as a result of the pandemic, and public perception of schools in its wake. Asked whether the steep drop in performance could be remedied with time and more resources, his answer was stark.

“The honest answer is that we don’t know, as we’ve never seen a decline of this size and scope before,” West said.

The interview was edited for length and clarity.

The 74: The public reaction to these results has been huge — almost surprisingly so, given that prior studies have indicated significant learning deficits resulting from the pandemic. Do you think NAEP’s role as the authoritative national exam just makes these trends un-ignorable?

Martin West: It has certainly been gratifying, as a member of the National Assessment Governing Board, to see the strong reaction to the results. And I do think that reaction speaks to NAEP’s reputation as the authoritative source for tracking the achievement of American students over time. The NAEP Long-Term Trend, in particular, has remained essentially unchanged for more than fifty years. NAEP assessments are also the only tests that are routinely administered to samples that are truly representative of the nation as a whole. I think the latter factor in particular helps explain why NAEP results garner so much national media coverage. Reports based on state tests are inherently local stories. Reports based on interim assessment data leave room for doubts about whether they are truly representative. NAEP releases are national stories.

Do you think the steady trickle of bad news, whether from NAEP or other tests, is related to the diminished perceptions of public schools found in the EdNext poll, among others?

I think it is impossible to separate the role bad news about test scores has played in shaping public perceptions from the role played by factors like prolonged school closures that caused test scores to decline in the first place. So the bad news is definitely related to the diminished perceptions of school quality, but I don’t think we can say for sure that it has an independent effect.

It sounds pretty bad for students to be scoring at levels that were last seen 20 or more years ago. But what should we expect in terms of bounce-back achievement now that schools are essentially all offering in-person learning? In other words, while this is a very sharp one-time decline, is it likely to reset the learning trajectory for millions of kids permanently?

The honest answer is that we don’t know, as we’ve never seen a decline of this size and scope before. Recent reports based on interim assessments suggest that, as students resumed in-person instruction, they have generally demonstrated rates of achievement growth that were typical before the pandemic. That is encouraging as far as it goes, but it would not be enough to help the students whose educations have been disrupted return to where they would have been absent the pandemic.

This is one reason why I think it is critical to be clear about what we mean by recovery. Over the next decade, as students whose learning was not disrupted by the pandemic begin to move through the K-12 system, I’d expect NAEP results to revert back to pre-pandemic levels. We might then be tempted to say that the system as a whole has bounced back. But there are roughly 50 million students whose educations were disrupted — including two of my sons — and I would not want us to declare success unless we’ve also helped the specific students who were impacted make up lost ground. We have an obligation to help them experience accelerated rather than typical growth going forward.

On the other hand, these large scoring drops are presumably also interacting with the long-term stagnation or declines that we saw in last year’s release of long-term trends data from before the pandemic. If the pre-COVID situation was essentially one of weak growth, is it fair to say that the mere return to in-person learning won’t be enough to get students back on track?

That’s exactly right. From 2009 to 2019, we’ve seen the unfortunate combination of stagnant average scores and growing inequality between higher- and lower-achieving students. Today’s release confirmed that those lower-achieving students were also hardest hit by the pandemic. A return to business as usual would therefore only reinforce the pandemic’s unequal effects rather than offset them.

My impression is that the Long-Term Trend results since the early ’70s have essentially shown slowly shrinking performance gaps between students in different subgroups. But yesterday’s release indicated that the math disparity between white and African American students is now growing, and I believe Hispanic and students in the National School Lunch Program (a common metric of poverty) also experienced larger declines in math than white and non-NSLP students, respectively. How concerned should we be that COVID has not only led to general learning loss, but also hindered the progress of historically disadvantaged subgroups?

One legitimate (if clearly partial) success story of American education that is well documented by the Long-Term Trend NAEP is the gradual narrowing of achievement gaps between racial and ethnic groups. It is therefore jarring to see the math gap between Black and white students increase sharply in this year’s data. The other differences you note across groups were not large enough to be statistically significant, but they do point in the same direction of greater inequality. This is not necessarily surprising given what we know about the pandemic’s impact on historically disadvantaged subgroups generally, but it is certainly concerning.

Were you as surprised as I was by the reading scores, which didn’t show widening racial gaps between white, Hispanic, and African American students? Given what existing studies have shown about literacy setbacks during the pandemic, I was expecting a different result. It also seemed noteworthy that schools in cities, which obviously enroll disproportionate percentages of non-white students, didn’t see lower reading scores for nine-year-olds. How much faith should we put in these figures?

I agree that this pattern in reading is a bit unexpected, all things considered, but I find it hard to be too encouraged by results showing an equally large decline across these three racial groups. I’ll also be curious to see if this pattern is confirmed on the “main NAEP” results set for release this October. The lack of a decline for city schools is also a puzzle. Here, though, it is important to keep in mind that the NCES [National Center for Education Statistics, which administers the NAEP test] definition of “city” schools is not limited to large urban districts. 

For example, my own local district in Newton, Massachusetts, is classified as a “city” district despite the fact that most Boston-area residents would think of it as a suburb. The results for the 26 school districts that participate in the Trial Urban District Assessment and will be included in the main NAEP release will provide a clearer picture of what’s happened in the nation’s big-city schools.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter