Push for single rating of LA schools challenges state's multi-dimensional accountability system

Credit: Flickr

Louis Freedberg

September 5, 2019

8 Comments

Republish

There is a deeply rooted impulse in American society — perhaps any society — to rank everything from restaurants and refrigerators to athletes and colleges.

That may help explain why pressures continue in California to rank its schools based on a single score of some kind, despite a major thrust in the state to move in the opposite direction.

Great Schools, the hugely popular parent-oriented website, has combined several indicators to come up with score between 1 and 10 for every school in the state.

Another notable push is coming from Los Angeles, where the state’s largest school district has been working over the past year on a plan to rank schools on a numerical 1-to-5 scale, a number that would be reached by combining students’ improvement on test scores and other factors.

But that plan is far from being implemented.  This week newly elected school board member Jackie Goldberg introduced a harshly critical resolution effectively calling on the district to abandon the idea in its current form. Among other assertions, she argued, “the value of a public school cannot be quantified in a single, summative rating, which can shame, penalize, or stigmatize schools, education professionals, students and entire communities.”

L.A. Unified’s efforts have echoes of California’s old Academic Performance Index, a single number usually ranging from 600 to over 800, assigned to every school and district based almost entirely on student scores on state and federally mandated tests. Using those numbers, schools were ranked on 1-to-10 scale, and to take into account the impact of the economic, racial and ethnic backgrounds of students, schools were ranked on a numerical scale comparing them to schools with similar student populations.

But in 2014 the state scrapped the API and the 1-to-10 rankings and in their place instituted an accountability system of “multiple measures” that include not only test scores, but also graduation rates, chronic absenteeism rates, suspension rates and harder-to-define indicators such as school climate and parent engagement.

According to a survey of the Education Commission of the States, all but five states have come up with a “summative” rating of some kind. In education testing parlance, that’s a single ranking that combines multiple indicators in the form of an A-F grade, a star or a tiered rating, like Massachusetts’ “Tier 1” to “Tier 4” scale.

With its multiple measures approach, California is definitely “bucking that trend,” Morgan Polikoff, a professor at USC’s Rossier School of Education, and Kate Kennedy, a research associate there, wrote in a paper last year.

Instead, the state  has developed the California School Dashboard, a relatively new system to rate schools using a complicated set of color codes ranging from red (the worst rating) to blue (the highest rating) on various indicators assigned to the school as a whole and numerous subgroups of students. But California schools are not assigned a single color to reflect their or their students’ overall performance.

Former State Board of Education President Michael Kirst, one of the chief architects of the new system, said there were many reasons not to do so — not least of which is that there is no scientific way to know what weights to assign to the different factors that make up a final rating. “What’s happened in the past and in other states is that they just grabbed numbers out of the air,” said Kirst, a nationally renowned Stanford University education scholar. “There’s no scientific basis for making these weightings and we couldn’t think of any way to do it.”

One of the main reasons for abandoning the API was the fact it didn’t capture everything happening in a school. As any parent or educator knows, so much more is going on in a school than just how students do on an annual standardized test. An overall ranking masks how subgroups of students are doing and doesn’t necessarily measure how students improved over time. It also didn’t measure at all what schools are doing to help students improve.

When the L.A. school board approved a resolution 18 months ago setting in motion a procedure to come up with an overall school ranking, its goal was to come up with “one comprehensive picture of school success.”

To its credit, the district is trying to come up with a measure that includes much more than test scores. In its draft form for high schools, 40 percent of L.A. Unified’s proposed ranking would be based on how much students’ scores improved over the previous year and 25 percent on the actual test scores. A measurement of “school climate” would make up another 20 percent of the ranking and a school’s success in preparing prepare students for college and careers the remaining 15 percent. Elementary and middle schools would have a similar approach.

Kirst said that this amalgam of measures would be an improvement over the old one-dimensional Academic Performance Index. But, he said, it still does not capture the nuances of what is happening in a school, or resolve the fundamental problem of how to weight the different measures that make up the ranking. “It’s just a number that is pulled out of the air in terms of its internal components,” he said.

L.A. Unified board member Kelly Gonez, a former teacher who is one of the ranking’s main backers, explained that its purpose is not to place blame or to be punitive, but to come up with a clearer view of schools with the greatest need, and to guide the district on where to best invest its resources.  “The goal is for us as a district and for the  broader community to get a sense of how our schools are doing, and how we can make sure that all schools are helping kids be prepared for college, careers and life,” she said in an interview on KPCC’s AirTalk.

For now, if only because of a change in the make-up of the board, the outlook for implementation of a new ranking system in Los Angeles is uncertain at best.

In her resolution attacking the whole idea, fellow board member Goldberg counters that “school ratings promote unhealthy competition between schools, exacerbate community antagonisms by producing artificial ‘winners’ and ‘losers’ and penalize schools that serve socio-economically disadvantaged students.”

Whatever happens in Los Angeles, Kirst points out that the California School Dashboard underwent a major revision last year, making it far more user-friendly than its first iteration. Other than in Los Angeles, “I don’t see any groundswell of objection,” he said.

California has put an enormous amount of effort and resources over the past five years into creating the dashboard and the  accountability system that it visually represents. The challenge for educators and elected officials — in Sacramento and at a local level — is whether they have the patience to give the new system time to work before jumping in and making changes to it, while at the same time assessing when and if reforms of the multi-layered system are necessary.

To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.