Stanford education professor Michael Kirst was a leading architect of California’s new accountability system based on multiple measures and the California School Dashboard that represents visually how schools are doing on numerous indicators. Kirst, a close adviser on education to Gov. Jerry Brown for several decades, was president of the State Board of Education when Brown became governor in 2010 and occupied a similar position during Brown’s first term as governor in the 1970s. Kirst has argued strongly against trying to rate a school or school district on a single measure. John Fensterwald and Louis Freedberg talked with Kirst to get his views on why he is opposed to a single rating. To hear an audio version of the interview in our podcast “This Week in California Education,” click here.
EdSource: Los Angeles Unified is talking about coming up with a system to rank its schools on a 1 to 5 scale. Do you still think that a single rating for schools is not the way to go and if so, why?
Michael Kirst: I strongly think that that’s not the way to go for both technical and policy reasons. On our California School Dashboard there are five or six state factors plus some local factors. To reduce it to a single factor, you would have to decide on a weight for each one. How much would be determined by test scores? Would that be 25%? 50%? How much would be determined by student suspensions? How much by graduation rates? How much by English Learner progress? What happened in the past and in other states is they just grabbed numbers out of the air. Maybe test scores ought to be 50% and graduation rates 10%? There’s no scientific basis for making these weightings and we couldn’t think of any basis for doing it that way.
Secondly, we also were very interested in measuring student growth year over year. How much would you weight growth from year to year versus weighting the actual test score in a particular year? There is just no empirical way to do this without making up arbitrary weights and numbers and getting a single number, which is misleading. It’s sort of like using the Dow Jones average for everything in the stock market. No reasonable investor would do that. So those are the technical reason we did it this way.
Thirdly, we wanted to set up a system of support for schools. Instead of a single number,we wanted to come up with a nuanced view of all of the many components and dimensions of the school around which we could design our system of support. A school may have good test scores, but its graduation rates may be terrible. It may have high test scores and poor English learner progress. We wanted to be more specific on how to intervene and help schools and support them.
EdSource: The single rating that Los Angeles Unified is talking about combines different indicators including some of the multiple measures on the California School Dashboard. It includes a measure of growth in student performance from one year to the next and how well schools are preparing students for college and careers. Isn’t that an improvement over the old system that was based just on test scores?
Kirst: It’s somewhat of an improvement. But I go back to how would they rate school climate? A lot of people think that should be given a low weight compared to test scores. Other people think social emotional learning ought to be given a high weight. If you have multiple components, you still have to reduce it all to a scale of one to five. It’s just a number that is pulled out of the air in terms of its internal components.
EdSource: If you put yourself in a place of a parent who wants to send their child to a magnet school in LA Unified, there are dozens of them. Wouldn’t a ranking at least be a start for a parent who then can look for more details if they wanted to on the Dashboard?
Kirst: We got the opposite advice from, for example, the California State PTA, which did not want to go to 1 to 5 scale and really likes our dashboard. And so it’s interesting to see who speaks for parents. Various groups claim to speak for parents, but the input we got was that parents realize there’s more to a school than rating it 1, 2, 3, 4 or 5. The overwhelming feedback we got was “tell us more about the nuances of how a school is doing.”
EdSource: The California School Dashboard has barely been in place a couple of years. At what point does the state start reforming this rather complex accountability system that you and others helped create? How much time do we want to give these reforms to work?
Kirst: There has been very little objection to the dashboard and mostly praise. I’m amazed at how well it has gone, especially once we issued it in a clearer form last year so that we now have a much better school report than we had in the first year. I don’t see any ground swell of objection. I sense that this is not something that is running into much of a headwind. This L.A. group is the first one I’ve seen that has really pushed back. And it is a long way from getting approved by the school board.
EdSource: But the school district has put quite a lot of work looking into this over the past year.
Kirst: If we were getting a lot of objections and a lot of views that we should go back to a single measure, then we would we need to take a hard look at this. But I am not hearing that.
EdSource: It seems like human nature to want to rank things. I mean, we have rankings of refrigerators and athletes and colleges. And so it seems like there is a deep-seated impulse to want to rank things. Do you think that’s part of what’s going on?
Kirst: Yes, I think that’s part of what’s going on and our role as educational leaders and as a state that wants to be on the front edge of things is to push back from that. We’ve noticed that a lot of states have withdrawn from an A to F ranking system that was so popular because their people have said it was just too simplistic.
To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.
We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.
Gregory Geeting 4 years ago4 years ago
So, just to be sure that Stanford University professors’ views are fully represented, are you going to interview Edward Haertel, who was a key advisor on the development of the Dashboard’s predecessor, the Academic Performance Index (API)?
Mary Ellen 4 years ago4 years ago
"A school may have good test scores, but its graduation rates may be terrible. It may have high test scores and poor English learner progress." Seriously? Does Kirst know what he's taking about? Good test scores correlate with high graduation rates. Poor English learner progress correlates with low test scores. Also: graduation rates, and especially, suspension rates are meaningless. For example, does a drop in suspension … Read More
“A school may have good test scores, but its graduation rates may be terrible. It may have high test scores and poor English learner progress.” Seriously? Does Kirst know what he’s taking about?
Good test scores correlate with high graduation rates. Poor English learner progress correlates with low test scores.
Also: graduation rates, and especially, suspension rates are meaningless. For example, does a drop in suspension rate mean that fewer students are misbehaving? Or, does it mean that bad behavior is tolerated by the administration (to the detriment of the other students)?
Dr. Bill Conrad 4 years ago4 years ago
Much ado about nothing! The primary mission of schools is to ensure the academic achievement of the students not to redirect the purpose in engineering social-emotional characteristics! It is so clear that the state of California and specifically Los Angeles Unified are failing the students. Just take a look at the data at http://sipbigpicture.com and you will quickly see the magnitude of this cataclysmic failure! Maybe its time to consider a transformation of K-12 education rather … Read More
Much ado about nothing!
The primary mission of schools is to ensure the academic achievement of the students not to redirect the purpose in engineering social-emotional characteristics!
It is so clear that the state of California and specifically Los Angeles Unified are failing the students. Just take a look at the data at http://sipbigpicture.com and you will quickly see the magnitude of this cataclysmic failure!
Maybe its time to consider a transformation of K-12 education rather than engage in useless diversions like accountability numbers as it only contributes to the fog of education!
Maybe start with a rebuild of the color in the lines colleges of education so that they recruit our finest and train them well starting with super emphasis on content knowledge and expert implementation of professional practices! Crazy idea! I know.
The system continues to work for apologist adults who are expert at diverting our attention to the myriad of inanities in K-12 education rather than the real mission of academic achievement through high quality aligned systemwide educator content knowledge and professional practices implementation!
The adults are just too happy to collect their 6-figure salaries and dither rather than to have the courage to initiate and sustain the required transformational changes. Too bad the accountability scales don’t reach into negative values because that is really where we are as a system in K-12 education especially for students of color and economically disadvantaged students!
Shame on us! Our kids and families deserve so much more than the current scam and con!