Early data on learning loss show big drop in math, but not reading skills

Photo by Allison Shelley for American Education: Images of Teachers and Students in Action

Two remote learning students collaborate on their schoolwork at home.

Photo by Allison Shelley for American Education: Images of Teachers and Students in Action

Two remote learning students collaborate on their schoolwork at home.

John Fensterwald

November 30, 2020

4 Comments

Republish

A national testing organization that predicted school closures last spring would leave students far behind where they should be academically as they entered schools this fall released data Tuesday showing the reality, while worrisome, was less dire.

The test results from the Portland-based nonprofit organization NWEA is one of the first large-scale efforts documenting the actual “learning loss” caused by the disruptions of Covid-19 that educators fear could set back a generation of students.

Other researchers also have tried to quantify “learning loss,” although the term may be a misnomer, at least in this case. While some students’ math and reading skills actually did regress on the organization’s MAP Growth assessment, most returned to school this fall showing gains from the spring — but less than what they otherwise would have achieved, NWEA said.

Test scores of 4.4 million students in grades 3 through 8 found students did lose significant ground overall in math but generally started school about where they should be in reading, the organization reported. NWEA (formerly known as the Northwest Evaluation Association) provides school districts with interim assessments they can choose to use at the start, midway and end of a school year to track students’ academic growth.

Although scores didn’t decline as much as anticipated, the results include a huge asterisk. The organization compared the scores of students in schools that gave the tests in both fall 2019 and fall 2020. But 25% of those students, predominately low-income Black and Hispanic students, didn’t take the tests this fall. Since these also are the students most disadvantaged by remote learning, NWEA called the results “incomplete” and probably underestimated how much students nationwide were set back last spring.

Analyses of the scores can’t explain why so many students missed fall 2020 assessments, NWEA stated. “However, two possibilities are that students may have opted out of testing because they lack reliable technology or because they have disengaged from school due to economic, health, or other factors,” the report said. “Either scenario presents an urgent call for intervention.”

The report did not provide data by states. In California, results from 65 school districts with 158,000 students were included. Primarily small districts and charter schools contract with NWEA, along with Santa Ana, the state’s ninth-largest district, with 47,000 students.

Last April, about a month after most school districts shut down for the year or turned quickly to remote learning, NWEA projected the worst-case scenario that assumed the remainder of the year would include little or no instruction. It would be comparable to a longer “summer melt” — the annual backslide that most students experience when on school vacation.

Students would return in the fall with 70% of the learning gains in reading and less than 50% of the gains in math compared with a typical school year.

Put a different way, a student who scored in the 50th percentile on fall 2019 test in math would be expected to score between the 40th and 30th percentile this fall.

Instead, the results were closer to about half as bad as projected: a drop of 5 to 10 percentile points, to between the 45th and 40th percentile, NWEA reported.

In Santa Ana Unified, the drop in learning from the fall of 2019 to the fall of 2020 was comparable to students nationwide. The decline in reading in the elementary grades was slightly larger, 2.5 to 4.4 percentile points depending on the grade. The decline in math scores was somewhat smaller, 2.5 percentile points in third grade, 8.4 percentile points in fourth grade, said Emily Wolk, Santa Ana’s director of research and evaluation. The participation rate in Santa Ana, a predominately low-income district, beat the national average, with 93% to 97% taking the fall 2020 tests in elementary grades, 81% to 87% grades 6 to 8.

The quick turnaround in getting results of the tests, which the district administered remotely this year — a challenge in itself — has helped teachers identify struggling students and intervene early. Scheduled office hours give teachers time to customize assistance and work in small groups, she said.

Usually, from year to year, the students’ NWEA scores are generally stable. Roughly the same proportion of students achieve the top and bottom scores. But compared with 2019, about twice the number of students nationwide had a significant decline in math scores, falling at least 20 percentile points from last spring to this fall, with the biggest drop in fourth-grade math.

In a press conference, Chris Minnich, CEO of NWEA, credited teachers for the continued progress in reading last spring, despite the pandemic, while acknowledging that it can be hard for parents to guide their kids’ learning in math. Remote learning can make it more difficult for teachers to identify students’ gaps in learning, and it’s more difficult to remediate outside the classroom, he said.

The disparities in scores varied from grade to grade, with nearly twice as big a drop in expected growth in third- and fourth-grade math as in seventh- and eighth-grade math. Students showed the least growth in fourth-grade reading scores, although the difference between last year and this year was only a couple of percentile points.

The CORE Data Collaborative, a nonprofit that tracks school improvement measures in hundreds of California districts, has also noticed a lower loss of learning among upper grades in a preliminary look at the impact of school closures over the past year, said Rick Miller, CORE’s executive director. The leading hypothesis, he said, is older students are better able to work independently and are more comfortable with technology.

But beyond tests results, there are other disturbing indicators of Covid’s impact on students’ mental health and their disconnection with school in California: higher rates of failing grades and of chronic absenteeism, in an analysis of 33 districts by the consulting firm SI&A.

Last spring, the consulting company McKinsey & Company made longer term projections of the impact of Covid closures. If schools remained closed intermittently and did not reopen fully until January 2021, it estimated that students could lose three to four months of learning if they received average remote instruction and seven to 11 months with lower quality remote instruction. Lower income students, Black and Hispanic students — those less likely to have a quiet space with minimal distractions, devices they do not need to share and parental academic supervision — would suffer the biggest losses, it predicted.

McKinsey also predicted more students would drop out. “The virus is disrupting many of the supports that can help vulnerable kids stay in school: academic engagement and achievement, strong relationships with caring adults and supportive home environments,” it said.

Miller said CORE’s initial look at the data of its sampling of districts shows it’s problematic to generalize about learning loss since “learning appears inconsistent, even within schools. That is to say, from classroom to classroom some appear more successful than others with distance learning,” he said.

“We want to better understand this and to document our areas of success,” he said.

To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.