California at bottom in nationwide ranking of accountability systems; state board president disagrees

Credit: Tim Gouw for unsplash

Credit: Tim Gouw for unsplash

John Fensterwald

November 14, 2017

11 Comments

Republish

Another prominent education research and advocacy organization that disapproves of California’s approach to school accountability has ranked California’s new system at the bottom nationwide in a report released Tuesday.

The low score by the Thomas B. Fordham Institute reflects a core disagreement over how best to identify and work with schools needing help. California education leaders are unapologetic about the route they’ve chosen, and they say the Fordham analysis contains a key error.

Like Bellwether Education Partners, which harshly criticized the state’s approach in an August analysis, Washington, D.C.- and Ohio-based Fordham gives high grades to states that will rank schools with an A-F letter grade or a similar method that’s understandable at a glance. States will use rankings to select the lowest-performing schools, as required by the federal Every Student Succeeds Act.

California’s color-coded school dashboard does not give a summary school ranking. Each measure of performance, whether test scores, graduation rates or student suspension rates, gets a separate color rating. Gov. Jerry Brown and the State Board of Education say that this approach focuses attention on specific areas that need work. While this is more complex — and, some critics say, confusing — advocates say it is more helpful in diagnosing problems.

Fordham also rated California poorly for not giving more weight to how much students’ test results improve year over year. And it criticized its current methodology to measure schools’ growth in scores.

In a statement, State Board of Education President Michael Kirst said Fordham’s ranking judged California’s system narrowly and failed to appreciate “the totality of California’s efforts to improve outcomes for all students.”

“This report arbitrarily picks out only 3 components of a large multi-component plan to rate states,” Kirst wrote in an email. “The California School Dashboard provides communities with unprecedented access to meaningful information about schools,” he wrote. “The Dashboard’s Equity Report makes achievement gaps impossible to ignore.” The equity report breaks down student performance by demography, race and ethnicity and other factors, highlighting disparities.

But in its report, Fordham praised the 34 states that used either A-F ratings or “user-friendly numerical systems.”

“Easy-to-understand labels, such as A–F letter grades, provide clear signals to parents, citizens, and educators about the quality of a school and can nudge systems toward improvement. ‘Dashboards’ — which present lots of data points but no bottom line — are great complements to labels, especially when teams sit down to determine how to take a school to the next level, but they are no replacement,” wrote authors Michael Petrilli, the institute’s president, and Brandon Wright, the institute’s editorial director.

California’s color scheme is a work in progress. Early next year, the state board will adopt a methodology for applying color ratings for multiple indicators on the dashboard to identify lowest-performing schools. So far, the board and the California Department of Education have found this to be harder to do than they anticipated.

The report examined states’ ESSA applications and based the rankings on whether states incorporated three criteria that the authors said were important for accountability. The new federal law requires that states identify the lowest-performing 5 percent of schools and measure their performance and improvement, starting in fall 2018. The three criteria are:

Did states assign annual ratings to schools that are clear and intuitive for parents, educators, and the public? In other words, use something like an A to F grade.

Did they encourage schools to focus on all students, not just their low performers? Under the previous No Child Left Behind law, states measured progress by the percentage of students who scored proficient in math and English language arts, compared with students in the same grade the year before. This led states to focus on “bubble kids,” those just below states’ proficiency cutoffs, Fordham wrote, instead of all students. To focus on all students’ improvement, Fordham said states should track the growth of individual students’ scores from year to year and create an index that credits gains below and above proficiency. Fordham gave 23 states an excellent rating.

Did they fairly measure and judge all schools, including those with high rates of poverty? Crediting improvement is critical, especially for high-poverty schools that might score low in achievement, the report said. Accurately assessing the true performance of schools “can’t be done unless it’s possible for high-performing, high-poverty schools to actually earn positive ratings,” the report said. To get a high rating, Fordham required academic growth to make up at least 50 percent of a state’s accountability system.

Seven states got high grades in all three categories. California, North Dakota and Idaho were the only three states to get low scores in all three.

In fairness to California, it probably deserved a grade of “Incomplete.” The state board’s goal is to adopt a true growth model, tied to the annual growth of individual students’ test scores, by fall of 2018, when its school accountability system will go into effect.

David Sapp, deputy policy director and assistant legal counsel for the state board, said the report also contained a big error. California already has moved away from the old standard of rating achievement based on the percentage of students who scored proficient. The dashboard measures performance in relation to the point identified as minimum proficiency on the Smarter Balanced math and English language arts tests. It measures how far above or below that point students, on average, scored.

“The solution is to adopt a growth model,” Petrilli said in an interview. “When it gets its act together, their rating would go up.”

Finally, instead of counting growth in scores for 50 percent of the basis of its accountability system, Fordham said it counts for 20 percent in California — hence its low score on the third criterion. But the study doesn’t state how it came up with the figure of 20 percent, and neither Kirst nor Sapp could figure that out.

It is true that test scores will count for less than 50 percent in the accountability system, but growth and improvement — Fordham’s goal — account for half of a school’s color rating on each academic and nonacademic indicator on the dashboard.

To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.