iStockphoto.com

Editor’s note: California officials are at a turning point as they begin a lengthy process of replacing the Academic Performance Index with a new system of measuring school and district performance. In this background report, EdSource editor-at-large John Fensterwald explains what the transformation might look like from the perspectives of key players involved.

The State Board of Education is seizing the chance to redefine student achievement and reframe how schools are held accountable for performance. It is in the throes of replacing the Academic Performance Index, the three-digit number that has been California’s narrow gauge of school progress for a decade and a half. The question is, what will take its place?

Fully rolling out a new accountability system is projected to take three years – there is no legislated deadline. But state board members and others who have shared their thoughts have expressed similar concepts of what it might – and should not – be.

There is near-universal agreement among educators and policy makers that a new system should be distinctly different from the API, which is calculated by weighting school and district scores on various subject assessments. Instead of a single number with consequences tied to end-of-year standardized tests, there should be multidimensional measures reflecting the complexities of school life and performance, including potentially hard-to-quantify indicators of school climate, as well as test scores and indicators of success in preparing students for college and career options. State board President Michael Kirst uses the analogy of gauges on a car dashboard that display oil pressure, temperature, battery capacity and mileage, each measuring different components of a car’s performance.

Although there are shades of difference, state board members and educators generally agree that school improvement should be the overriding goal of a new accountability system and that schools and districts should be given time and flexibility to achieve specific and clear goals. This approach would contrast with a decade of top-down sanctions under the federal No Child Left Behind law, said Rick Simpson, deputy chief of staff for Assembly Speaker Toni Atkins and education adviser to a half-dozen Assembly speakers. “As a state, we kind of reached the consensus that the mechanisms of intervention and punishment were not effective tools for changing behavior,” he said.

A new accountability system would culminate a series of historic changes that are already reshaping K-12 education in California. These started with the shift of authority and responsibility over budgets and policy from the state to local school districts under a new funding system that directed more money to low-income children and students learning English. The funding law established Local Control and Accountability Plans, or LCAPs, that require districts to set goals and steer money to meet broader indicators of school performance than test scores alone can provide. New academic standards – the Common Core State Standards and Next Generation Science Standards – have raised expectations and shifted attention to the complex challenges of preparing students to succeed beyond high school.

Jannelle Kubinec of WestEd explains evaluation rubrics at a conference in January sponsored by the research organization PACE.

PACE

Jannelle Kubinec of WestEd explains evaluation rubrics at a conference in January sponsored by the research organization PACE.

A new accountability system must tie these big educational shifts together, says Jannelle Kubinec, who directs the Comprehensive School Assistance Program for the San Francisco-based research agency WestEd and also is leading outreach efforts for the state board. “We are at a rare moment of change,” she said at a recent conference in Sacramento on the future of school accountability. “If we blow it now, we are not going to have this opportunity again, at least with regard to aligning policy.”

David Plank, executive director of the independent research organization Policy Analysis for California Education, agrees about the magnitude of the potential changes.

“California has embarked on something radical,” he said. “The state is moving dramatically from top-down standardized accountability to something with a much smaller state role and a much smaller role for test scores. After having spent the last 20 years getting folks to sing from the same hymnal, we’re now saying, ‘Do what is best for your district and community.’”

In January, Plank and Stanford School of Education professor Linda Darling-Hammond published a policy paper, “Supporting Continuous Improvement in California’s Education System,” that laid out the principles for a new accountability system built around the goals of giving districts the flexibility and teachers the resources to create an environment that fosters continuous improvement. Kirst praises the report for providing a framework for what the system might look like while acknowledging it could end up looking quite different.

For starters, there’s no agreement yet among key players – groups representing teachers, local school boards, members of the state board, legislative leaders and especially civil rights organizations – on the details. There is also no consensus on where to strike a balance between giving districts breathing room to improve on their own and upholding the state’s obligation to see that all students get the opportunity for a quality education. Finally, there’s uncertainty whether Congress, in rewriting the No Child Left Behind law – if and when that happens – will give states the flexibility to devise their own approaches to accountability that California is counting on.

Plank said that in the decades before the advent of a state-based accountability system and No Child Left Behind, “there was a long history in which local control worked to the detriment of poor and Latino kids.” Advocacy groups are concerned that if turned loose, “districts will screw kids again,” he said. “Then there are people like me who say, ‘I have seen generations of policy failure (under prescriptive state and federal mandates), so it’s time to try something different.’”

The state API and the No Child Left Behind law are both test-based systems. The API was created in 1999 to determine which schools deserved extra money through the Governor’s Performance Awards and which needed state oversight through the Immediate Intervention/Underperforming Schools Program. The state suspended both programs after a few years but continued to calculate an annual API. Penalties under No Child Left Behind have been meted out to schools and districts based on percentages of students who failed to attain proficiency, mainly on state English language arts and math tests. Sanctions include giving students the right to attend better-performing schools and setting aside a portion of a school’s federal money for tutoring.

Many dimensions to a school

The state board is not starting from scratch in designing a new system. In the 2013 law establishing LCAPs, Gov. Jerry Brown and legislative leaders signaled they favored a multidimensional approach. They created eight priority areas for school improvement that the LCAPs must address and two dozen sources of data that districts must use to track progress. The priorities include student engagement, as measured by middle and high school dropout rates, attendance rates, absenteeism and graduation rates; school climate, as measured by suspension and expulsion rates and student surveys; implementation of standards, including Common Core; and student achievement. Metrics for the latter include rates of reclassifying English learners as proficient in English and passing Advanced Placement exams, scores on standardized tests, and, finally, a school’s API scores. Lawmakers assumed the API would continue to exist when they wrote the law.

Rick Simpson and Sue Burr discuss the future of accountability at a conference sponsored by Policy Analysis for California Education in Sacramento in January.

PACE

Rick Simpson and state board member Sue Burr discuss the future of accountability at a conference sponsored by Policy Analysis for California Education in Sacramento in January.

Listing two dozen metrics that districts must track in their LCAPs does not by itself constitute a state accountability system that sets uniform targets for achievement and consequences for failing to meet them. The Legislature, however, did rough out a structure for one and gave the state board the job of fleshing it out. It will begin to take shape over the next year as the board addresses three issues: the future of the API, evaluation metrics and the role of the new state agency formed to monitor improvement and oversee assistance for persistently troubled schools.

The future of the API

This spring, parents and schools will receive the results of standardized tests on the Common Core standards in math and English language arts. The state board must decide whether test scores, broken down by school and student subgroup, should stand on their own as one component of the new accountability system. Or the board could give the scores more prominence through a reconstituted API that would eventually incorporate yet-to-be-developed tests in science and social studies, a high school exit exam aligned to new standards and perhaps career and college readiness indicators.

A state advisory committee is recommending replacing the API, and Plank also favors eliminating it, to end what he views as the distorted power that legislators and realtors (who price houses based on a neighborhood school’s API) give to an imperfect, one-dimensional measure of achievement. But that won’t be easy or non-controversial. The API is embedded throughout state education laws, and the Legislature might have to rewrite or expunge them. That won’t happen this year, Kirst said. In a letter to the state board, a coalition of two dozen civil rights and children’s advocacy organizations said they favor keeping the API “to ensure California will have in place a single coherent system of support, assistance, and intervention.”

Last year, the state board suspended the API so that school districts could focus full attention on implementing the new Common Core State Standards and preparing for new computer-based Smarter Balanced tests on the standards that students will take this spring. The Legislature has given the board the option of suspending the API a second year as well. There’s a good chance that the board will do so when it meets on March 11 in order to buy more time for statutory changes.

How many metrics?

The Legislature recognized that an accountability system needs a refined set of tools, so it gave the state board until Oct. 1, 2015 to create a set of “evaluation rubrics” that fulfill a broad set of purposes.  The law says the state board should set performance targets and goals for improvement for all of the metrics that apply to each of the eight LCAP priorities. The board would set, say, a statewide target that 60 percent of high school seniors would pass the 15 courses, known as A-G, for admission to the University of California or the California State University systems; currently about  40 percent of seniors satisfy the A-G course requirement. The board would then list the expected rates of improvement for districts, schools and a dozen student subgroups.

Since last fall, Kubinec of WestEd has been meeting with education groups and holding public meetings across the state to gather views on the evaluation rubrics. Last Friday, as part of the state board’s agenda for the March meeting, WestEd released a draft of what the evaluation rubrics might look like (see a sample from page 8 of the draft, below). The organization and display of the data are patterned after the accountability system used by the Canadian province of Alberta.

 

A multi-colored display would show whether districts, schools and student subgroups are or aren't  meeting the state or locally defined target for each metric in the evaluation rubric, such as high school graduation rates, access to career or college prep courses, performance on standardized tests. This shows a small portion of the data display of a draft evaluation rubrics proposed for the State Board by the research agency WestEd.

Credit: WestEd

A multi-colored display would show whether districts, schools and student subgroups are or aren’t meeting the state or locally defined target for each metric in the evaluation rubric, such as high school graduation rates, access to career or college prep courses, performance on standardized tests. This shows a small portion of the data display of a draft evaluation rubrics proposed for the State Board by the research agency WestEd.

The state law says that the evaluation rubrics should serve several purposes:

  • Help charter schools and school districts identify strengths and weaknesses;
  • Help county offices of education, which must approve district LCAPs, identify weaknesses that districts must act on to show improvement;
  • Assist the state superintendent of public instruction to identify those districts that, having failed to improve, require intervention.

The evaluation rubrics will function as both a guide for district self-improvement and a basis for determining outside intervention. The state board will have to spell out how the two purposes will mesh and how the rubrics will help, not duplicate or conflict with, the LCAPs that districts spend months preparing. WestEd’s draft rubrics include sections in which districts will be asked to analyze what is and isn’t working and identify strategies and best practices they’ll adopt.

The board must answer many questions in establishing metrics and establishing a process for holding schools accountable for meeting them. Among them:

Should there be uniform evaluation metrics for all districts, or could districts devise them, based on their own preferences and priorities?

Burr and Kirst said they’d be open to a mix of state and local metrics. Particularly in the areas of career readiness and student engagement, districts could serve as laboratories for experimentation, Kirst said. The civil rights coalition took a harder line in an analysis of a preliminary version of WestEd’s rubrics. “We believe the (Local Control Funding Formula) statute requires that the State Board establish uniform state performance and improvement standards. Anything less than uniform statewide standards would undercut, if not irreparably impair, meaningful accountability for ensuring equality of educational opportunity, improving student outcomes and closing the achievement gap for all students.”

What and how many measurements of school performance will all districts and charter schools be held accountable for?

The state board could require lots of them. But board member Sue Burr, the board’s liaison in the evaluations rubrics process, said she views the evaluations rubrics primarily as a tool for districts’ self-assessment. For state accountability purposes, she favors holding districts accountable for perhaps no more than three metrics required in the evaluation rubrics. An accountability system should not resemble a checklist under the No Child Left Behind law, she said. “We want locals to embrace their responsibilities,” she said. John Affeldt, managing attorney of the nonprofit law firm Public Advocates and one of the authors of the civil rights coalition’s analysis, agreed in principle but said the coalition hasn’t yet taken a position on what would trigger intervention. “That’s a separate conversation,” he said.

Who will determine what assistance schools that aren’t improving should receive, and how much time districts should be given to meet a state target before technical assistance or more intrusive interventions are imposed?

Under No Child Left Behind, schools that failed to meet proficiency targets were labeled Program Improvement schools, which was next to impossible to escape from, and the federal government gave them a narrow choice of interventions. Schools resented the process. State board members and legislative leaders say they want a constructive process, starting with county offices of education, which would review the districts’ evaluation rubrics and suggest forms of expertise and assistance. The civil rights coalition suggested a timetable of five to seven years for schools and student subgroups to make performance targets. But the requirement for “opportunity to learn” metrics, like ensuring that students have textbooks and qualified teachers, should be “absolute” and immediate, Affeldt said.

Should there be one model of school accountability or local variations for charter schools, schools serving students at risk of dropping out or schools that focus on careers?

Seven California school districts already are creating their own novel school accountability system, with a multiple-measure index, under a one-year U.S. Department of Education waiver from the No Child Left Behind law. Called the CORE districts and encompassing nearly a million students, the seven would like to continue to use their measures, as long as they can show schools are improving, CORE executive director Rick Miller said recently.

Role of a new state agency

A small agency with a big mission will make the final decision on which schools and districts will need assistance and ultimately state intervention. Created two years ago with a $10 million initial budget, the California Collaborative for Educational Excellence held its first meeting on Wednesday. It will determine which consultants, county offices and school leaders have the expertise in areas such as  parent engagement, career and technical education or working with English language learners and then assign them to schools needing help. It also will decide if a persistently low-performing school or district needs state oversight.

The Legislature signaled its intentions when it chose the agency’s name. It should be a place where districts go to collaborate for assistance and referrals; intervention should be the last resort, Simpson said. Burr, who is serving as the state board’s representative on the collaborative’s five-member board, said its first role should be a “repository of learning” that collects and disseminates best practices.

In writing the rules for intervention, however, the state board will also determine when and how often the collaborative must take a harder line on intervention.

When she talks about the new accountability model, Burr uses terms that have seemed foreign during the era of No Child Left Behind.

One is “trust.” “We have a huge task to rebuild trust to say we are serious about local accountability,” she said. “We have to trust that local educators will do what is right on behalf of students they serve.”

Another is “patience.” “It will take a long time to build a coherent system in a meaningful way,” she said.

Yet another is “adaptability.” “We need to build in self-reflection and examine goals and the evaluation rubrics every five years or so,” she said. “We never did this with the API.”

But what one person views as needed patience, another person views as needless inaction. Ted Lempert, president of the advocacy group Children Now, said he worries that the pendulum may swing too far in response to past problems.

“There’s no question our previous accountability system was flawed,” he wrote in an email. “There’s also no question that it shined a light on the achievement gap and was critical to the gains we’ve seen among kids of color over the last decade plus. In constructing the new system, California leaders shouldn’t overreact to past flaws by shying away from the state’s role in ensuring that all students and their progress are tracked each year.”

That tension will shape what the next school accountability system looks like.

 

To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.

Share Article

Comments (7)

Leave a Comment

Your email address will not be published. Required fields are marked * *

Comments Policy

We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.

  1. Brian Silberberg 9 years ago9 years ago

    The thinking behind this article makes a ton of sense; we need to think creatively and divergent to ensure we have a fulfilling and effective education system in place. At Books That Grow that’s exactly what we’re committed to. Our app provides a library of reading materials, each of which can be read across multiple reading levels, to ensure all students can have their needs met in the classroom simultaneously. See more at http://www.booksthatgrow.com/

  2. Don 9 years ago9 years ago

    When local control advocates say to do what's best for the local district I wonder to what they are specifically referring? All districts are bound by a litany of various state and federal mandates. They all have to teach CCSS and take the SB assessments. They all have the same voluminous ed code to abide. They mostly have similar language and demographic challenges. They all have to function within the confines of collectively bargained contracts. … Read More

    When local control advocates say to do what’s best for the local district I wonder to what they are specifically referring? All districts are bound by a litany of various state and federal mandates. They all have to teach CCSS and take the SB assessments. They all have the same voluminous ed code to abide. They mostly have similar language and demographic challenges. They all have to function within the confines of collectively bargained contracts. And they all, with a few exceptions, are hard pressed to conform to the above with the funding made available.

    When CCEE, with a budget of a few million and a diminutive staff, steps in to advise – what is it, do you suppose, they can offer that experienced administrators and teachers haven’t already thought of in terms of greater accountability? What magic bullet do they possess? I understand the new accountability is more of a carrot than a stick, but it has yet to be made clear how the CCEE will function in its capacity to advise, particularly since they only managed to meet for the first time last week.

    As the Jamaican-American poet Claud McKay said:

    “Idealism is like a castle in the air if it is not based on a solid foundation of social and political realism.”

    The CDE is out of the business of compliance and instead they have delegated this monumental task of overseeing the largest public school system to a sorely under-resourced surrogate in the form of CCEE.
    The buck now stops with the local board of education or the court.

  3. Paul Muench 9 years ago9 years ago

    Seems like the new system is best named an improvement system and not an accountability system. The outcome of the measurements I can identify is that the California Collaborative for Educational Excellence will direct advisers to districts that need improvement. Is there any other outcome for having measures that reflect the need for improvement?

    Replies

    • navigio 9 years ago9 years ago

      This is an important comment. This is the distinction I think Burr was trying to highlight. Under the current system, the way to 'improve' a failing school/district is to get different students. This goes hand in hand with parents prioritizing peer environment over teacher or school 'quality'. In that sense, we need to ask ourselves why we want 'accountability'. If the goal is improvement then we likely need to do a lot differently. Read More

      This is an important comment. This is the distinction I think Burr was trying to highlight.
      Under the current system, the way to ‘improve’ a failing school/district is to get different students. This goes hand in hand with parents prioritizing peer environment over teacher or school ‘quality’. In that sense, we need to ask ourselves why we want ‘accountability’. If the goal is improvement then we likely need to do a lot differently.

      • Don 9 years ago9 years ago

        Like closing the revolving door of teachers at hard-to-staff schools and ameliorating the negative effects of home life for some by configuring a longer school day, having extracurricular activities in the middle of the day with classes and staffed study halls and tutoring in the later afternoon as well as lunch and dinner provided. The check, please.

        • FloydThursby1941 9 years ago9 years ago

          Agreed, let principals decide whether or not they wish to continue the employment of teachers who have been teaching a long time and don't have good reputations. Let children be the first priority, teachers second. Let every teacher be as nervous as other professions so they think twice about calling in sick when perfectly healthy. Guaranteeing any profession step salary increases and virtuay no chance of being fired even if they work … Read More

          Agreed, let principals decide whether or not they wish to continue the employment of teachers who have been teaching a long time and don’t have good reputations. Let children be the first priority, teachers second. Let every teacher be as nervous as other professions so they think twice about calling in sick when perfectly healthy. Guaranteeing any profession step salary increases and virtuay no chance of being fired even if they work at an average level and take maximum days off each year without fail, even if they have complaints, is a guarantee of mediocrity This is a universal truth which is not seriously disputed.

  4. navigio 9 years ago9 years ago

    Education is politics. As a result, much of our state or fed driven policy will be contradictory and conflicting. We've had almost 2 decades of the API and we still cant figure out what it means. The confusion seems to stem from two areas: that there are multiple and conflicting priorities the number is hoping to communicate; and that the basis for the number largely represents things that have nothing to do with school/district behavior … Read More

    Education is politics. As a result, much of our state or fed driven policy will be contradictory and conflicting.

    We’ve had almost 2 decades of the API and we still cant figure out what it means. The confusion seems to stem from two areas: that there are multiple and conflicting priorities the number is hoping to communicate; and that the basis for the number largely represents things that have nothing to do with school/district behavior (though people may still care about them for different reasons).
    In order to clarify that confusion, the individual components must be looked at and their causes evaluated (and discussed!) independently. This requirement is unlikely to be any different for any ‘new system’ than it was for the ‘old system’, which (obviously) conflicted/conflicts with the public’s ‘demand’ for simplistic measures, and for that reason, is/will be unlikely to happen (One of Burr’s comments seems to acknowledge this conflict). As a result, the only likely outcome to have any chance of satisfying most people is to provide a laundry list of rubrics and let those who care enough about understanding them do so and (try to) explain them to others. Hiding those things and hoping no one notices, while we also hold the apparent assumption that, unchecked, districts will not behave in the best interests of their students, does not seem like a useful approach.
    An alternative is of course to trust schools districts, but that’s unlikely to happen anytime soon.