Moving beyond a single measure of a school

March 1, 2015

Editor’s note: California officials are at a turning point as they begin a lengthy process of replacing the Academic Performance Index with a new system of measuring school and district performance. In this background report, EdSource editor-at-large John Fensterwald explains what the transformation might look like from the perspectives of key players involved.

The State Board of Education is seizing the chance to redefine student achievement and reframe how schools are held accountable for performance. It is in the throes of replacing the Academic Performance Index, the three-digit number that has been California’s narrow gauge of school progress for a decade and a half. The question is, what will take its place?

Fully rolling out a new accountability system is projected to take three years – there is no legislated deadline. But state board members and others who have shared their thoughts have expressed similar concepts of what it might – and should not – be.

There is near-universal agreement among educators and policy makers that a new system should be distinctly different from the API, which is calculated by weighting school and district scores on various subject assessments. Instead of a single number with consequences tied to end-of-year standardized tests, there should be multidimensional measures reflecting the complexities of school life and performance, including potentially hard-to-quantify indicators of school climate, as well as test scores and indicators of success in preparing students for college and career options. State board President Michael Kirst uses the analogy of gauges on a car dashboard that display oil pressure, temperature, battery capacity and mileage, each measuring different components of a car’s performance.

Although there are shades of difference, state board members and educators generally agree that school improvement should be the overriding goal of a new accountability system and that schools and districts should be given time and flexibility to achieve specific and clear goals. This approach would contrast with a decade of top-down sanctions under the federal No Child Left Behind law, said Rick Simpson, deputy chief of staff for Assembly Speaker Toni Atkins and education adviser to a half-dozen Assembly speakers. “As a state, we kind of reached the consensus that the mechanisms of intervention and punishment were not effective tools for changing behavior,” he said.

A new accountability system would culminate a series of historic changes that are already reshaping K-12 education in California. These started with the shift of authority and responsibility over budgets and policy from the state to local school districts under a new funding system that directed more money to low-income children and students learning English. The funding law established Local Control and Accountability Plans, or LCAPs, that require districts to set goals and steer money to meet broader indicators of school performance than test scores alone can provide. New academic standards – the Common Core State Standards and Next Generation Science Standards – have raised expectations and shifted attention to the complex challenges of preparing students to succeed beyond high school.


Jannelle Kubinec of WestEd explains evaluation rubrics at a conference in January sponsored by the research organization PACE.

A new accountability system must tie these big educational shifts together, says Jannelle Kubinec, who directs the Comprehensive School Assistance Program for the San Francisco-based research agency WestEd and also is leading outreach efforts for the state board. “We are at a rare moment of change,” she said at a recent conference in Sacramento on the future of school accountability. “If we blow it now, we are not going to have this opportunity again, at least with regard to aligning policy.”

David Plank, executive director of the independent research organization Policy Analysis for California Education, agrees about the magnitude of the potential changes.

“California has embarked on something radical,” he said. “The state is moving dramatically from top-down standardized accountability to something with a much smaller state role and a much smaller role for test scores. After having spent the last 20 years getting folks to sing from the same hymnal, we’re now saying, ‘Do what is best for your district and community.’”

In January, Plank and Stanford School of Education professor Linda Darling-Hammond published a policy paper, “Supporting Continuous Improvement in California’s Education System,” that laid out the principles for a new accountability system built around the goals of giving districts the flexibility and teachers the resources to create an environment that fosters continuous improvement. Kirst praises the report for providing a framework for what the system might look like while acknowledging it could end up looking quite different.

For starters, there’s no agreement yet among key players – groups representing teachers, local school boards, members of the state board, legislative leaders and especially civil rights organizations – on the details. There is also no consensus on where to strike a balance between giving districts breathing room to improve on their own and upholding the state’s obligation to see that all students get the opportunity for a quality education. Finally, there’s uncertainty whether Congress, in rewriting the No Child Left Behind law – if and when that happens – will give states the flexibility to devise their own approaches to accountability that California is counting on.

Plank said that in the decades before the advent of a state-based accountability system and No Child Left Behind, “there was a long history in which local control worked to the detriment of poor and Latino kids.” Advocacy groups are concerned that if turned loose, “districts will screw kids again,” he said. “Then there are people like me who say, ‘I have seen generations of policy failure (under prescriptive state and federal mandates), so it’s time to try something different.’”

The state API and the No Child Left Behind law are both test-based systems. The API was created in 1999 to determine which schools deserved extra money through the Governor’s Performance Awards and which needed state oversight through the Immediate Intervention/Underperforming Schools Program. The state suspended both programs after a few years but continued to calculate an annual API. Penalties under No Child Left Behind have been meted out to schools and districts based on percentages of students who failed to attain proficiency, mainly on state English language arts and math tests. Sanctions include giving students the right to attend better-performing schools and setting aside a portion of a school’s federal money for tutoring.

Many dimensions to a school

The state board is not starting from scratch in designing a new system. In the 2013 law establishing LCAPs, Gov. Jerry Brown and legislative leaders signaled they favored a multidimensional approach. They created eight priority areas for school improvement that the LCAPs must address and two dozen sources of data that districts must use to track progress. The priorities include student engagement, as measured by middle and high school dropout rates, attendance rates, absenteeism and graduation rates; school climate, as measured by suspension and expulsion rates and student surveys; implementation of standards, including Common Core; and student achievement. Metrics for the latter include rates of reclassifying English learners as proficient in English and passing Advanced Placement exams, scores on standardized tests, and, finally, a school’s API scores. Lawmakers assumed the API would continue to exist when they wrote the law.


Rick Simpson and state board member Sue Burr discuss the future of accountability at a conference sponsored by Policy Analysis for California Education in Sacramento in January.

Listing two dozen metrics that districts must track in their LCAPs does not by itself constitute a state accountability system that sets uniform targets for achievement and consequences for failing to meet them. The Legislature, however, did rough out a structure for one and gave the state board the job of fleshing it out. It will begin to take shape over the next year as the board addresses three issues: the future of the API, evaluation metrics and the role of the new state agency formed to monitor improvement and oversee assistance for persistently troubled schools.

The future of the API

This spring, parents and schools will receive the results of standardized tests on the Common Core standards in math and English language arts. The state board must decide whether test scores, broken down by school and student subgroup, should stand on their own as one component of the new accountability system. Or the board could give the scores more prominence through a reconstituted API that would eventually incorporate yet-to-be-developed tests in science and social studies, a high school exit exam aligned to new standards and perhaps career and college readiness indicators.

A state advisory committee is recommending replacing the API, and Plank also favors eliminating it, to end what he views as the distorted power that legislators and realtors (who price houses based on a neighborhood school’s API) give to an imperfect, one-dimensional measure of achievement. But that won’t be easy or non-controversial. The API is embedded throughout state education laws, and the Legislature might have to rewrite or expunge them. That won’t happen this year, Kirst said. In a letter to the state board, a coalition of two dozen civil rights and children’s advocacy organizations said they favor keeping the API “to ensure California will have in place a single coherent system of support, assistance, and intervention.”

Last year, the state board suspended the API so that school districts could focus full attention on implementing the new Common Core State Standards and preparing for new computer-based Smarter Balanced tests on the standards that students will take this spring. The Legislature has given the board the option of suspending the API a second year as well. There’s a good chance that the board will do so when it meets on March 11 in order to buy more time for statutory changes.

How many metrics?

The Legislature recognized that an accountability system needs a refined set of tools, so it gave the state board until Oct. 1, 2015 to create a set of “evaluation rubrics” that fulfill a broad set of purposes.  The law says the state board should set performance targets and goals for improvement for all of the metrics that apply to each of the eight LCAP priorities. The board would set, say, a statewide target that 60 percent of high school seniors would pass the 15 courses, known as A-G, for admission to the University of California or the California State University systems; currently about  40 percent of seniors satisfy the A-G course requirement. The board would then list the expected rates of improvement for districts, schools and a dozen student subgroups.

Since last fall, Kubinec of WestEd has been meeting with education groups and holding public meetings across the state to gather views on the evaluation rubrics. Last Friday, as part of the state board’s agenda for the March meeting, WestEd released a draft of what the evaluation rubrics might look like (see a sample from page 8 of the draft, below). The organization and display of the data are patterned after the accountability system used by the Canadian province of Alberta.


Credit: WestEd

A multi-colored display would show whether districts, schools and student subgroups are or aren’t meeting the state or locally defined target for each metric in the evaluation rubric, such as high school graduation rates, access to career or college prep courses, performance on standardized tests. This shows a small portion of the data display of a draft evaluation rubrics proposed for the State Board by the research agency WestEd.

The state law says that the evaluation rubrics should serve several purposes:

The evaluation rubrics will function as both a guide for district self-improvement and a basis for determining outside intervention. The state board will have to spell out how the two purposes will mesh and how the rubrics will help, not duplicate or conflict with, the LCAPs that districts spend months preparing. WestEd’s draft rubrics include sections in which districts will be asked to analyze what is and isn’t working and identify strategies and best practices they’ll adopt.

The board must answer many questions in establishing metrics and establishing a process for holding schools accountable for meeting them. Among them:

Should there be uniform evaluation metrics for all districts, or could districts devise them, based on their own preferences and priorities?

Burr and Kirst said they’d be open to a mix of state and local metrics. Particularly in the areas of career readiness and student engagement, districts could serve as laboratories for experimentation, Kirst said. The civil rights coalition took a harder line in an analysis of a preliminary version of WestEd’s rubrics. “We believe the (Local Control Funding Formula) statute requires that the State Board establish uniform state performance and improvement standards. Anything less than uniform statewide standards would undercut, if not irreparably impair, meaningful accountability for ensuring equality of educational opportunity, improving student outcomes and closing the achievement gap for all students.”

What and how many measurements of school performance will all districts and charter schools be held accountable for?

The state board could require lots of them. But board member Sue Burr, the board’s liaison in the evaluations rubrics process, said she views the evaluations rubrics primarily as a tool for districts’ self-assessment. For state accountability purposes, she favors holding districts accountable for perhaps no more than three metrics required in the evaluation rubrics. An accountability system should not resemble a checklist under the No Child Left Behind law, she said. “We want locals to embrace their responsibilities,” she said. John Affeldt, managing attorney of the nonprofit law firm Public Advocates and one of the authors of the civil rights coalition’s analysis, agreed in principle but said the coalition hasn’t yet taken a position on what would trigger intervention. “That’s a separate conversation,” he said.

Who will determine what assistance schools that aren’t improving should receive, and how much time districts should be given to meet a state target before technical assistance or more intrusive interventions are imposed?

Under No Child Left Behind, schools that failed to meet proficiency targets were labeled Program Improvement schools, which was next to impossible to escape from, and the federal government gave them a narrow choice of interventions. Schools resented the process. State board members and legislative leaders say they want a constructive process, starting with county offices of education, which would review the districts’ evaluation rubrics and suggest forms of expertise and assistance. The civil rights coalition suggested a timetable of five to seven years for schools and student subgroups to make performance targets. But the requirement for “opportunity to learn” metrics, like ensuring that students have textbooks and qualified teachers, should be “absolute” and immediate, Affeldt said.

Should there be one model of school accountability or local variations for charter schools, schools serving students at risk of dropping out or schools that focus on careers?

Seven California school districts already are creating their own novel school accountability system, with a multiple-measure index, under a one-year U.S. Department of Education waiver from the No Child Left Behind law. Called the CORE districts and encompassing nearly a million students, the seven would like to continue to use their measures, as long as they can show schools are improving, CORE executive director Rick Miller said recently.

Role of a new state agency

A small agency with a big mission will make the final decision on which schools and districts will need assistance and ultimately state intervention. Created two years ago with a $10 million initial budget, the California Collaborative for Educational Excellence held its first meeting on Wednesday. It will determine which consultants, county offices and school leaders have the expertise in areas such as  parent engagement, career and technical education or working with English language learners and then assign them to schools needing help. It also will decide if a persistently low-performing school or district needs state oversight.

The Legislature signaled its intentions when it chose the agency’s name. It should be a place where districts go to collaborate for assistance and referrals; intervention should be the last resort, Simpson said. Burr, who is serving as the state board’s representative on the collaborative’s five-member board, said its first role should be a “repository of learning” that collects and disseminates best practices.

In writing the rules for intervention, however, the state board will also determine when and how often the collaborative must take a harder line on intervention.

When she talks about the new accountability model, Burr uses terms that have seemed foreign during the era of No Child Left Behind.

One is “trust.” “We have a huge task to rebuild trust to say we are serious about local accountability,” she said. “We have to trust that local educators will do what is right on behalf of students they serve.”

Another is “patience.” “It will take a long time to build a coherent system in a meaningful way,” she said.

Yet another is “adaptability.” “We need to build in self-reflection and examine goals and the evaluation rubrics every five years or so,” she said. “We never did this with the API.”

But what one person views as needed patience, another person views as needless inaction. Ted Lempert, president of the advocacy group Children Now, said he worries that the pendulum may swing too far in response to past problems.

“There’s no question our previous accountability system was flawed,” he wrote in an email. “There’s also no question that it shined a light on the achievement gap and was critical to the gains we’ve seen among kids of color over the last decade plus. In constructing the new system, California leaders shouldn’t overreact to past flaws by shying away from the state’s role in ensuring that all students and their progress are tracked each year.”

That tension will shape what the next school accountability system looks like.


Exit mobile version