A national testing organization that predicted school closures last spring would leave students far behind where they should be academically as they entered schools this fall released data Tuesday showing the reality, while worrisome, was less dire.
The test results from the Portland-based nonprofit organization NWEA is one of the first large-scale efforts documenting the actual “learning loss” caused by the disruptions of Covid-19 that educators fear could set back a generation of students.
Other researchers also have tried to quantify “learning loss,” although the term may be a misnomer, at least in this case. While some students’ math and reading skills actually did regress on the organization’s MAP Growth assessment, most returned to school this fall showing gains from the spring — but less than what they otherwise would have achieved, NWEA said.
Test scores of 4.4 million students in grades 3 through 8 found students did lose significant ground overall in math but generally started school about where they should be in reading, the organization reported. NWEA (formerly known as the Northwest Evaluation Association) provides school districts with interim assessments they can choose to use at the start, midway and end of a school year to track students’ academic growth.
Although scores didn’t decline as much as anticipated, the results include a huge asterisk. The organization compared the scores of students in schools that gave the tests in both fall 2019 and fall 2020. But 25% of those students, predominately low-income Black and Hispanic students, didn’t take the tests this fall. Since these also are the students most disadvantaged by remote learning, NWEA called the results “incomplete” and probably underestimated how much students nationwide were set back last spring.
Analyses of the scores can’t explain why so many students missed fall 2020 assessments, NWEA stated. “However, two possibilities are that students may have opted out of testing because they lack reliable technology or because they have disengaged from school due to economic, health, or other factors,” the report said. “Either scenario presents an urgent call for intervention.”
The report did not provide data by states. In California, results from 65 school districts with 158,000 students were included. Primarily small districts and charter schools contract with NWEA, along with Santa Ana, the state’s ninth-largest district, with 47,000 students.
Last April, about a month after most school districts shut down for the year or turned quickly to remote learning, NWEA projected the worst-case scenario that assumed the remainder of the year would include little or no instruction. It would be comparable to a longer “summer melt” — the annual backslide that most students experience when on school vacation.
Students would return in the fall with 70% of the learning gains in reading and less than 50% of the gains in math compared with a typical school year.
Put a different way, a student who scored in the 50th percentile on fall 2019 test in math would be expected to score between the 40th and 30th percentile this fall.
Instead, the results were closer to about half as bad as projected: a drop of 5 to 10 percentile points, to between the 45th and 40th percentile, NWEA reported.
In Santa Ana Unified, the drop in learning from the fall of 2019 to the fall of 2020 was comparable to students nationwide. The decline in reading in the elementary grades was slightly larger, 2.5 to 4.4 percentile points depending on the grade. The decline in math scores was somewhat smaller, 2.5 percentile points in third grade, 8.4 percentile points in fourth grade, said Emily Wolk, Santa Ana’s director of research and evaluation. The participation rate in Santa Ana, a predominately low-income district, beat the national average, with 93% to 97% taking the fall 2020 tests in elementary grades, 81% to 87% grades 6 to 8.
The quick turnaround in getting results of the tests, which the district administered remotely this year — a challenge in itself — has helped teachers identify struggling students and intervene early. Scheduled office hours give teachers time to customize assistance and work in small groups, she said.
Usually, from year to year, the students’ NWEA scores are generally stable. Roughly the same proportion of students achieve the top and bottom scores. But compared with 2019, about twice the number of students nationwide had a significant decline in math scores, falling at least 20 percentile points from last spring to this fall, with the biggest drop in fourth-grade math.
In a press conference, Chris Minnich, CEO of NWEA, credited teachers for the continued progress in reading last spring, despite the pandemic, while acknowledging that it can be hard for parents to guide their kids’ learning in math. Remote learning can make it more difficult for teachers to identify students’ gaps in learning, and it’s more difficult to remediate outside the classroom, he said.
The disparities in scores varied from grade to grade, with nearly twice as big a drop in expected growth in third- and fourth-grade math as in seventh- and eighth-grade math. Students showed the least growth in fourth-grade reading scores, although the difference between last year and this year was only a couple of percentile points.
The CORE Data Collaborative, a nonprofit that tracks school improvement measures in hundreds of California districts, has also noticed a lower loss of learning among upper grades in a preliminary look at the impact of school closures over the past year, said Rick Miller, CORE’s executive director. The leading hypothesis, he said, is older students are better able to work independently and are more comfortable with technology.
But beyond tests results, there are other disturbing indicators of Covid’s impact on students’ mental health and their disconnection with school in California: higher rates of failing grades and of chronic absenteeism, in an analysis of 33 districts by the consulting firm SI&A.
Last spring, the consulting company McKinsey & Company made longer term projections of the impact of Covid closures. If schools remained closed intermittently and did not reopen fully until January 2021, it estimated that students could lose three to four months of learning if they received average remote instruction and seven to 11 months with lower quality remote instruction. Lower income students, Black and Hispanic students — those less likely to have a quiet space with minimal distractions, devices they do not need to share and parental academic supervision — would suffer the biggest losses, it predicted.
McKinsey also predicted more students would drop out. “The virus is disrupting many of the supports that can help vulnerable kids stay in school: academic engagement and achievement, strong relationships with caring adults and supportive home environments,” it said.
Miller said CORE’s initial look at the data of its sampling of districts shows it’s problematic to generalize about learning loss since “learning appears inconsistent, even within schools. That is to say, from classroom to classroom some appear more successful than others with distance learning,” he said.
“We want to better understand this and to document our areas of success,” he said.
To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.
We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.
Doug McRae 3 years ago3 years ago
The NWEA data for 4.4 million students included only 158,000 students from CA, only 3-4 percent of the total data (presumably including 47,000 from Santa Ana), not enough data to make reasonable inferences for CA as a whole. The CORE data is much more robust for early indications of learning loss. Likely better data would be Smarter Balanced interim test data, but the CDE only releases "tests taken" data info (in aggregate in the millions of … Read More
The NWEA data for 4.4 million students included only 158,000 students from CA, only 3-4 percent of the total data (presumably including 47,000 from Santa Ana), not enough data to make reasonable inferences for CA as a whole. The CORE data is much more robust for early indications of learning loss.
Likely better data would be Smarter Balanced interim test data, but the CDE only releases “tests taken” data info (in aggregate in the millions of students for CA) rather than “test results” information. Presumably test results for Smarter Balanced interim tests would provide a better information base for early CA data on learning loss. The Smarter Balanced consortium computers collect SBAC interim test results information, but the consortium will only release that information when authorized by CDE.
Dr. Bill Conrad 3 years ago3 years ago
The failure to ensure that 1/4 of the population, including mostly children of color, take the test invalidates any legitimate interpretation of the data.
By interpreting and reporting results that are woefully incomplete is not supportable and racist as it implies that the performance of the children of color really doesn’t matter.
That is the real flaw.
Dr. Bill Conrad 3 years ago3 years ago
I would like to exhort my fellow educators to take a few moments to donate to EdSource. The great team at EdSource has been working diligently to keep us informed on all matters educational during the pandemic. We rely on EdSource for accurate information and commentary. Please donate and help them reach their goal! Kudos to those school districts who were able to administer the MAP interim assessment in math and reading! I am especially impressed … Read More
I would like to exhort my fellow educators to take a few moments to donate to EdSource.
The great team at EdSource has been working diligently to keep us informed on all matters educational during the pandemic. We rely on EdSource for accurate information and commentary. Please donate and help them reach their goal!
Kudos to those school districts who were able to administer the MAP interim assessment in math and reading! I am especially impressed with Santa Ana School District for completing the MAP test remotely.
As is its nature, COVID-19 continues to expose the racism endemic within the K-12 education system. The state and our school districts must make sure that all of our students and especially our children of color have access to quality technology and broadband.
It is not OK to throw up our hands and walk away.
We cannot depend upon handouts from the tech industry. Access to technology is mission critical in this new era. We must reorganize budgets to ensure that all students have access to technology and broadband.
It may mean that staff has to make significant efforts to reach out to the families of our children of color to make sure that they have the technology that they need to continue their children’s education including interim testing.
We need to end this blatant racism and do what is necessary to support our children of color.
Enough is enough.
Christopher A Ross 3 years ago3 years ago
There goes California again. Leaving out minorities in its numbers. Online has been a disaster. But last year didn’t count, the district assures parents. Mismanaged budgets before pandemic. Teachers refusing to go back to teaching kids. California is a disgrace.