Initial data for California confirm early grades, low-income children hit hardest by ‘learning loss’

First graders answer questions for a project on the white board.

Credit: Allison Shelley for American Education

The first California study measuring declines in learning during the first months of the pandemic parallels findings nationally: There has been a significant drop in test results in the early grades, with low-income students and English learners showing the least progress in learning.

The university-based research organization PACE released the results Monday. While finding “substantial learning loss” overall in both English language arts and math, it said average numbers “mask the reality that some students in California are suffering much more during this time than are others.”

Over the course of a school year students are expected to learn new content and skills. Learning loss measures how much students fell short of the typical annual growth. The PACE study compares the fall 2020 scores with scores from fall 2019 of 50,000 students in grades 4 to 10 from 18 unnamed school districts in California. It measures what a typical year-to-year gain in tests aligned to the state’s academic standards would be compared with the results in the pandemic.

All 18 districts are part of the CORE Data Collaborative, a collection of 200 districts that examine student growth in measurements beyond what the state collects. Six of the districts took what’s called the MAP assessment administered by NWEA, a Portland-based creator of assessments, formerly known as the Northwest Evaluation Association; the other 12 took the Star assessments administered by the K-12 testing company Renaissance. Both tests are interim assessments, given two or three times to measure progress over the course of a year.

Both have previously released national findings based on their fall-to-fall assessments. NWEA’s research included 65 California school districts, primarily small districts and charter schools.

Results of the two tests are treated separately, since the MAP assessment is more than twice as long, measures different standards and uses different point scales. But on both assessments, there was a sizable learning loss in the early grades in math and reading, with the most differences in fifth and sixth grades. At the same time, there was unanticipated growth beyond the expected gains in the upper grades, especially in ninth- and 10th-grade math on the MAP test, and ninth-grade math and 10th-grade English language arts on the STAR test.

Further study might reveal whether these gains reflect true accelerated learning or could be attributed to older students’ facility with technology, changes in the test, a drop in student participation or student cheating, the study said.

Except for citing a few examples, the study did not translate the losses and gains in points and scale scores into percentage gains and losses.

The losses and gains in points and scale scores for each grade don’t readily translate into percentage gains and losses, but the authors did calculate a few: On the MAP fourth-grade and eighth-grade English language arts test, low-income students showed a 7% decline in the normal rate of learning, while wealthier students showed a 5% faster than normal growth rate. Combining the two rates equal a one-year learning gap of 12%. On the fifth-grade MAP English language arts test, English language learners showed 30% less growth than normal, while non-English learners showed only a 10% drop — a gap of 20 percentage points.

The authors noted limitations of the study and said further studies would delve into more details and examine the breakdown by racial and ethnic groups. Because a disproportionate number of low-income and special education students in the districts did not take the latest test, the results could understate the severe impact that the pandemic has had on those students, the study said.

The tests were administered from August through October and so primarily reflected learning loss from March to the return to school, when most districts acknowledged they were thrown into distance learning unprepared. A winter 2021 assessment could reveal better data — reflecting that districts have provided more students with internet access and teachers have mastered skills in distance learning instruction, said Heather Hough, executive director of PACE and an author of the study.

Most of the districts presumably were in distance learning; further analysis will show whether some had in-person instruction or offered hybrid instruction involving both in-person and distance learning.

The study noted that the demographic makeup of the students in the study was diverse but not an exact replica of California. There was a larger proportion of Latino students, English language learners and low-income students in the 18 districts than the state as a whole. This might explain why, for example, the national NWEA research found significant learning loss in math but not reading skills in the early grades, while the California districts found a relative decline in both areas.

In November, EdSource cited the results of Santa Ana Unified in its article on the nationwide NWEA research. Hough confirmed that Santa Ana was also included in the PACE study. The other 17 districts were smaller districts.

Although PACE’s initial findings are preliminary, the authors felt it was important to release the findings now, when the Legislature is considering decisions affecting students’ return to school this year, summer school and the design of school next fall. That includes how to allocate funding for the students struggling most.

Newsom is proposing to target most of the $4.6 billion in state funding for extended learning and the return to school to English learners and low-income students — the student groups that the PACE study found have been falling the farthest behind during distance learning.

“Without aggressive and bold actions, these students may never catch up; any funding or support designed to mitigate learning loss must be targeted specifically to the students who need it most,” the study concluded.