Credit: Alison Yin/EdSourceAt their meeting on May 12, members of the State Board of Education are expected to finally adopt what other states have adopted and what’s been under study for years in California: a way to include individual students’ progress on state standardized tests as part of the state’s school accountability system.
State board members agree with the criticism of advocates of student equity and school accountability hawks that a “student growth model” should replace what the state now uses to measure student achievement on the California School Dashboard. That current method compares the test results of the latest 4th graders with the previous year’s 4th grades to calculate change as an element on the dashboard.
Other states track the scores of individual students over time to measure if they are progressing or regressing. The U.S. Department of Education pressed California to adopt a similar growth model, too, when it approved the state’s plan for complying with the Every Student Succeeds Act. The state responded that it was in the works. For a number of years, the CORE Districts Data Collaborative has used a student growth model it developed for dozens of member districts.
There are several ways to measure students’ growth, and the state board chose what’s called a “residual gain” model. It calculates differences between students’ predicted test scores and actual test scores, using previous English language arts and math scores, as well as the scores of all other students in the same grade. But skeptical that the residual growth model would prove reliable and stable from year to year, the state board hired the test contractor ETS to make technical adjustments. It did, and technical reviewers for the state board are recommending the revised model be adopted.
The history and technical details are explained in detail in Item 4 of the board’s meeting agenda, which was posted last Friday.
The new growth model will be applied to English language arts and math test scores in grades 4 to 8. It can’t be used for 3rd grade because it’s the first year that students take Smarter Balanced tests (growth models need at least a year of growth) or for the 11th grade test results, because students don’t take Smarter Balanced in 9th or 10th grades.
One other wrinkle: Because of the pandemic, no students took Smarter Balanced tests in 2020, and many districts may not give it this year, under an agreement reached with the federal government. As a result, the first calculations using the new student growth model, incorporating three consecutive years of testing, won’t be released until December 2024, at the earliest. That will mark nearly a decade since the California Department of Education first looked at the issue, in 2015.
To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.
Comments (9)
Comments Policy
We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.
navigio 2 years ago2 years ago
Why did they choose to limit the the model to grades 4-8? And what is the technical hurdle in trying to apply the same methodology to grades 8 vs 11 without testing in between?
Replies
John Fensterwald 2 years ago2 years ago
navigio, the growth model requires the use of a a student’s previous year’s score. Since the first year to take tests is 3rd grade, the comparison would start with 4th grade. There is no Smarter Balanced test administered in 9th and 10th grades, which creates a gap in the data, complicating its application to 11th grade.
navigio 2 years ago2 years ago
Thanks John. Yes I understand that the goal is to have previous years’ data set available for the metric. What I was trying to ask is what about that process makes it so inaccurate that it won’t work past 1 year, even with some caveats. I read some of the documents but did not see a lower level discussion.
Brenda Lebsack 2 years ago2 years ago
Luis- concerning your comment about “no accountability for public schools for the exception of charter schools that are evaluated every 5 years”, you bring up an interesting point. As a public school teacher, I’m well aware that politicians are highly influenced by the teachers union. Bills continue to pass to make it harder and harder for charters to exist, while charters have the highest accountability standards. If a charter school under … Read More
Luis- concerning your comment about “no accountability for public schools for the exception of charter schools that are evaluated every 5 years”, you bring up an interesting point. As a public school teacher, I’m well aware that politicians are highly influenced by the teachers union. Bills continue to pass to make it harder and harder for charters to exist, while charters have the highest accountability standards. If a charter school under performs, it risks an approved renewal in 5 years, thus its very existence is threatened. If a traditional public school fails to perform, it is put on “school improvement” and receives supplemental resources from the state to help carry out those improvements. Does this seem like a double standard?
A new law, AB 1316 is now seeking to expand that disparity by another 3-5%. When it comes to equity and academic outcomes for students of all races and socio-economic stratus, I don’t think that’s the goal for our state educational bureaucrats.
https://californiaglobe.com/section-2/ab-1316-aims-to-destroy-charter-schools/
Replies
Bruce William Smith 2 years ago2 years ago
Brenda, these educrats live in an alternative reality in which they believe that they can achieve academic equity for students of all classes, if they just get the data & systems right, in spite of evidence spanning their entire careers that they cannot do so; instead of their schooling, we should follow Luis in demanding choice for families with respect to their children's education, in particular so they can access schools whose standards, in mathematics … Read More
Brenda, these educrats live in an alternative reality in which they believe that they can achieve academic equity for students of all classes, if they just get the data & systems right, in spite of evidence spanning their entire careers that they cannot do so; instead of their schooling, we should follow Luis in demanding choice for families with respect to their children’s education, in particular so they can access schools whose standards, in mathematics and other subjects, match those of the world’s educational leaders, rather than those of the failed Common Core, to which California remains attached.
Paul Muench 2 years ago2 years ago
It’s good to see that the California Board of Education is so optimistic. We’ll now be prepared when this level of precision is actually needed.
Luis 2 years ago2 years ago
This is a good thing. But, all Californians need to know that school districts have zero pressure to get better. There is no accountability in public schools, except for charter schools that get evaluated every 5 years. Now, under this woke union leadership mentality, student growth, or lack of, means absolutely nothing. Poor black and brown kids have been suffering through this for decades and the system has done nothing. This will only get worse … Read More
This is a good thing. But, all Californians need to know that school districts have zero pressure to get better. There is no accountability in public schools, except for charter schools that get evaluated every 5 years.
Now, under this woke union leadership mentality, student growth, or lack of, means absolutely nothing. Poor black and brown kids have been suffering through this for decades and the system has done nothing. This will only get worse now with current leadership, and the post-Covid slide.
You all should ask for family choice before we lose more than just the black and brown crowd! I know that sounds ugly, but its reality… just look at the pre-Covid data.
Jim 2 years ago2 years ago
Maybe I am missing something but why can’t they use historical data to simulate what the metrics would have been if they used it? There should be plenty of longitudinal student test data in several of our larger districts.
Replies
Doug McRae 2 years ago2 years ago
Jim, using historical statewide data was done. When the first recommendation for a "residual gain" growth model was presented to the SBE in spring 2018 based on two years of Smarter Balanced data, the board questioned the stability of the model over multiple years, and modeling was done for a 3rd year. It showed the model was not stable for a 3rd year. So, ETS's technical folks went to work and developed adjustments to … Read More
Jim, using historical statewide data was done. When the first recommendation for a “residual gain” growth model was presented to the SBE in spring 2018 based on two years of Smarter Balanced data, the board questioned the stability of the model over multiple years, and modeling was done for a 3rd year. It showed the model was not stable for a 3rd year. So, ETS’s technical folks went to work and developed adjustments to the basic “residual gain” model, and showed the adjustments corrected the model to generate sufficient stability over 3 years, using Smarter Balanced data from 2017-18-19 as I recall.
Growth metrics following students from one grade to the next grade has been on a “wish list” for large scale K-12 test developers for at least 60 years, with lots of different models and assumptions, but they always have pros and cons. On the “con” side of the “residual gain with adjustments” model now being considered by SBE, the model requires student statewide test data for 2 or 3 years in succession to be included in growth model calculations; it excludes all other students, and mobility is one of the characteristics of schools and districts serving low wealth areas, English learners, etc. So, one criticism is that growth model results reflect only non-mobile students, no mobile students included. In addition, growth models involve statistical complexity on top of statistical complexity, and are very difficult to explain to folks without substantial statistical training.
However, growth models are now popular in many states, and in my opinion they do not clearly provide better gain information than traditional cross-sectional same grade comparisons (i.e., 4th graders this year compared to 4th graders previous years, etc). That paradigm reflects the reality of the challenges facing teachers and administrators serving large numbers of low wealth students, English learners, etc, to educate all of their students. So, while I’m not against growth models per se, I don’t at this time think they deserve their popularity.