Stagnant test results on the third year of the Smarter Balanced tests are posing an unexpected challenge for the State Board of Education: Hundreds of school districts may require county assistance because they failed to raise their scores in English language arts and math on the tests they took last spring.
As a result, at its meeting Wednesday, the state board will consider substantially reworking the criteria for rating schools’ and school districts’ scores. Doing so will reduce the number of poorly performing districts that could trigger outside help this year.
Without rejiggering the criteria for determining high and low performers, the number of districts in the “red” zone, the lowest performing level on the California School Dashboard, in English language arts would more than double, from 81 districts identified last spring in the trial run of the dashboard covering 2015-16 test results, to 169 for the 2016-17 tests. That would comprise 11 percent of school districts.
In math, districts that would be rated as red would nearly double from 119 identified for the 2015-16 tests, to 231 for the latest test — 14 percent of districts. Which districts won’t be known until the state releases more data later this month.
These numbers apply to average scores of all students in a district. They don’t include poor performance by low-performing student groups, including students with disabilities. The same dashboard criteria would apply to them, too. County offices of education could have to provide assistance to many additional school districts to help them raise those subgroups’ achievement.
Faced with that prospect, the California Department of Education is recommending that the state board reduce the number of districts, student subgroups and schools falling in the lowest performing level by changing the dashboard’s color coding and making other changes for test scores. The education department memo does not mention the capacity of 52 county offices of education to provide help. But in previous discussions of the dashboard, state board members said they were concerned that hundreds of districts needing help could overwhelm counties’ ability to provide it, particularly in the first year of a new accountability system.
The department is making its recommendation on advice from the Technical Design Group, a committee of technical advisers who have not been identified and who meet behind closed doors. In a Nov. 3 letter to the state board, 14 student advocacy and civil rights organization in the Local Control Funding Formula Equity Coalition criticized the board’s haste in deciding what to do. Later this month, the state is scheduled to formally launch the state’s long-awaited school and district accountability system, with the release of the second-year results of the dashboard.
“All this has come at the last minute, with very little notice to stakeholders and the public, a rushed closed-door meeting with the Technical Design Group, and no time to run analyses on how these proposed changes affect student subgroup performance. We should not be changing expectations for schools and districts without more data and understanding,” the letter said.
“The mode of operation employed at this point (by the Technical Design Group) is deeply troubling,” wrote Samantha Tran, senior management director at the Oakland-based children’s advocacy nonprofit Children Now, in a letter to the board. Children Now asked for more openness, with the state listing design group members and publicly posting its agendas and allowing the public to comment.
How the dashboard works
The dashboard uses color codes to rate the performance of schools, districts and a dozen student subgroups on a range of statewide indicators, including suspension rates, graduation rates, performance of English learners and Smarter Balanced test scores. Indicators for chronic absenteeism and students’ readiness for college and careers will be added in the future.
The five color performance bands — red, orange, yellow, green and blue, from lowest to highest — will determine consequences. Generally, districts that rank red in two or more indicators, such as suspension rates and test scores, must receive technical help from their county office of education. Districts also must highlight red or orange indicators in their Local Control and Accountability Plans and detail plans for improvement.
Low-performing schools won’t receive assistance from county offices until next fall, under criteria that the state board will choose early in 2018 under the federal Every Student Succeeds Act.
A district’s dashboard color for math and English language arts performance gives equal weight to two factors: the latest average student scores and the amount the scores grew or fell from the previous year.
In the 2015-16 Smarter Balanced tests, California’s students showed strong growth in the percentage of students who were proficient in English language arts and math from the initial year of testing. In designing the dashboard criteria based on that year’s results, the state built growth into the dashboard assumptions. But in 2016-17, state scores on average were essentially flat. And more low-performing schools showed significant declines in scores than the year before and fell from orange to red on the dashboard.
Administrators from the Department of Education are recommending countering that by changing parameters to increase the number of districts that would be green and orange while reducing the criteria for red. Low-scoring districts whose scores declined significantly would become orange, instead of red as they do under the current criteria.
In making its case, the department argued that doubling the number of red districts in the current methodology “does not meet the intended purpose of the accountability system, which is to establish goals that are ambitious but also attainable by all schools throughout the state.” The memo suggested that the state may have to adjust the dashboard criteria for the testing indicator again in the next few years to further smooth out variations.
In its letter, the Equity Coalition acknowledged that it’s not good to have accountability measures that fluctuate annually. “However, we do not think the answer is to establish the significant negative precedent of altering the rubrics to produce the result the State finds more palatable.” Children Now has proposed technical fixes that the advising technical experts did not consider.
Because Smarter Balanced scores were flat or fell last year in English language arts in all 14 states that gave the test, some experts in assessment, including Edward Haertel, professor emeritus at the Stanford Graduate School of Education (see end of article), and Doug McRae, a retired assessment executive for textbook publishers, have said the problem could lie with the design and content of the test itself, not with the lackluster performance of California students on it.
“Before modifying the accountability system so significantly, the State should have the Technical Design Group and its experts examine the (Smarter Balanced) test more closely,” the Equity Coalition wrote.
To address a lack of growth in scores in the Smarter Balanced test, “we have to address the deficiencies” in the test, “not tinker with” the deficiencies by changing the dashboard parameters and color schemes, McRae wrote to the state board.
Administrators for the testing consortium so far have dismissed criticisms of the test and defended its reliability.