Black teachers: How to recruit them and make them stay
Lessons in higher education: What California can learn
Keeping California public university options open
Superintendents: Well-paid and walking away
The debt to degree connection
College in prison: How earning a degree can lead to a new life
(The article was updated on July 17 to include a comment from San Ramon Valley Superintendent Mary Shelton.)
Students Matter, which is challenging tenure and other teacher protection laws in Vergara v. California, filed a second school lawsuit Thursday. It is suing 13 school districts that it claims are violating the state law requiring student scores on state standardized tests be a component of a teacher’s evaluation.
The lawsuit, filed in Contra Costa County Superior Court, says that the districts illegally agreed in contracts negotiated with teachers to exclude test scores. Students Matters is asking the court to order the districts to follow the Stull Act, as the 1971 law is known.
Students Matter is a nonprofit created and funded by David Welch, a Silicon Valley entrepreneur. In a decision last year, it won the first round in Vergara, when a state Superior Court judge ruled that five state laws establishing teacher layoffs by seniority, a two-year probationary period leading to tenure and procedures for dismissing teachers violated students’ constitutional rights. The case is on appeal.
Attorneys for Students Matter say that Doe v. Antioch, their new case, is the logical next step. “In Vergara v. California, we proved that teacher quality is the most important in-school factor affecting student success,” Theodore Boutrous said in a press release. “Meaningful and reliable teacher evaluations are crucial to determining whether students are being taught by effective teachers.”
Students Matter argues that student progress on state tests is an important and reliable measure of effective teaching. The Legislature agreed, when, in 1999, it amended the Stull Act to include test scores on state tests among the multiple measures in evaluations. The lawsuit says that in violating the law, the 13 school districts “intentionally disregard valuable student achievement data that are accessible to them, choosing instead to remain ignorant as to the quality of the teachers in their schools.”
“Meaningful and reliable teacher evaluations are crucial to determining whether students are being taught by effective teachers.” – Theodore Boutrous
The 13 districts serve approximately 250,000 students. Along with Antioch Unified, they are Chaffey Joint Union High School District, Chino Valley Unified, El Monte City School District, Fairfield-Suisun Unified, Fremont Union High School District, Inglewood Unified, Ontario-Montclair School District, Pittsburg Unified Saddleback Valley Unified, San Ramon Valley Unified, Upland Unified School District, and Victor Elementary School District.
The issue of test scores has been litigated already. In a 2012 ruling, Los Angeles County Superior Court Judge James Chalfant agreed with Sacramento-based nonprofit EdVoice’s contention that the Stull Act requires the use of standardized test scores and ordered Los Angeles Unified to start using them. Chalfant left it to the district and United Teachers Los Angeles to agree on how and to what extent test scores should factor in an evaluation. With the departure of Superintendent John Deasy last year, the district and UTLA restarted discussions and have yet to reach a deal.
Chalfant’s ruling applied only to L.A. Unified and, since it was not appealed, gained little traction. In a report released earlier this year that sampled 26 districts’ compliance with the Stull Act, EdVoice found that 13 ignored the requirement to use student test scores in evaluations, and teacher contracts in San Ramon Valley and Upland – both named in the latest suit – explicitly prohibited using them. Others partially complied with the law, according to EdVoice.
Mary Shelton, superintendent of San Ramon Valley Unified, said Friday that she could not comment on the lawsuit, which she had not seen. She said that the district’s existing contract includes conflicting passages. One clause says that scores from any tests cannot be used for evaluations; another says that teacher evaluations should consider the progress that students have made toward meeting academic standards.
Shelton said that the contract is being revised. The district and teachers union have agreed to piloting an evaluation system that includes evidence of student progress. “It is headed in the right direction,” she said.
In reaction to the Vergara ruling, legislators in both parties introduced several bills this year to rewrite the Stull Act. All would clarify the standards for evaluating teachers, better designate who needs improvement and increase the frequency of evaluations. Republicans’ versions were defeated along party lines. Two similarly worded bills authored by the chairs of the Assembly and Senate Education Committees – Assemblyman Patrick O’Donnell, D-Long Beach, and Sen. Carol Liu, D-La Cañada-Flintridge, have stalled amid opposition from education management and civil rights groups.
One of the contentious unresolved issues involves how much test scores would count – and who would decide. The Stull Act leaves it up to districts to determine whether scores should count a lot or a little. The latest versions of Liu’s and O’Donnell’s bills – AB 575 and SB 499 – would, like the Stull Act, require student achievement as one factor of an evaluation, and it would require test scores on state tests to be one factor of measuring achievement. Both bills, however, would require districts to negotiate all of the details with teachers. Groups like the Association of California School Administrators and the California School Boards Association strenuously oppose this change.
A grassroots campaign recalled two members of the Orange Unified School District in an election that cost more than half a million dollars.
Legislation that would remove one of the last tests teachers are required to take to earn a credential in California passed the Senate Education Committee.
Part-time instructors, many who work for decades off the tenure track and at a lower pay rate, have been called “apprentices to nowhere.”
A bill to mandate use of the method will not advance in the Legislature this year in the face of teachers union opposition.
Comments (64)
Comments Policy
We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.
Gary Ravani 9 years ago9 years ago
I always appreciate those who construct a well turned phrase or a tight and concise way of expressing a key idea. I always referred to the cottage industry, well funded often by actual industry titans, dedicated to bad mouthing or otherwise undermining the public schools as the "public school criticism industrial complex." The late and revered John Mockler said it much better and concisely with : "the schools suck industry." John went on to eviscerate … Read More
I always appreciate those who construct a well turned phrase or a tight and concise way of expressing a key idea. I always referred to the cottage industry, well funded often by actual industry titans, dedicated to bad mouthing or otherwise undermining the public schools as the “public school criticism industrial complex.” The late and revered John Mockler said it much better and concisely with : “the schools suck industry.” John went on to eviscerate the assertions of said “industry” with solid facts over the years operating from his position of one of CA’s foremost education policy experts.
When we get to teachers’ evaluations and the use of student test scores i think I will rely on the concise words of an actual journalist, Valerie Strauss, who, in one quick paragraph eviscerates the concepts of test based evaluations and VAM specifically:
“For years, many prominent researchers have warned against using VAM. They include a 2009 warning by the Board on Testing and Assessment of the National Research Council of the National Academy of Sciences, which stated that “VAM estimates of teacher effectiveness should not be used to make operational decisions because such estimates are far too unstable to be considered fair or reliable.” The Educational Testing Service’s Policy Information Center has said there are “too many pitfalls to making causal attributions of teacher effectiveness on the basis of the kinds of data available from typical school districts,” and Rand Corp. researchers have said that VAM results “will often be too imprecise to support some of the desired inferences.”
For those few interested in the details supporting her assertions (not to mention the NRC, ETS, and RAND) you can find the original article (with links) at the WAPO under the title: “Arne Duncan’s reaction to new research slamming teacher evaluation method he favors.”
BTW, the article goes on to say that Arne was little impressed by the findings of actual education and research experts that lined up against his evaluation proposals central to RTTT. I know, I was shocked to learn this too.
Gary Ravani 9 years ago9 years ago
Yet more information about the wellbeing of chidden in the US that is being hidden behind the litigious curtain by self-interested billionaires and self-appointed school "reformers." By all means, no one who fears being shocked by reality and having their opinions about the poor, conditions of poverty, or effects of poverty on schools should ever look behind that curtain. The Annie E Casey Foundation just released its annual report showing that the real way children have … Read More
Yet more information about the wellbeing of chidden in the US that is being hidden behind the litigious curtain by self-interested billionaires and self-appointed school “reformers.” By all means, no one who fears being shocked by reality and having their opinions about the poor, conditions of poverty, or effects of poverty on schools should ever look behind that curtain.
The Annie E Casey Foundation just released its annual report showing that the real way children have been “left behind,” is by the economy. Childhood poverty rates in the US, already some of the worst in the industrialized world, have increased by 4 percentage points between 2008 and 2013 (last year for complete data). The rates for Blacks and Native Americans are nearly double that for other groups, though Hispanics are also high. Poverty rates for Whites have declined slightly.
Particularly troublesome, according to the Foundation director , is the number of children confined to poor communities. This parallels other studies showing increased economic segregation, geographic segregation, and school segregation rates that mirror those found before Brown v Board.
The report notes that high school graduation rates are at an all time high and more kids have health coverage. These improvements are not enough to offset the “particularly pernicious form of cancer” represented by abject poverty according to the Foundation director.
The report also notes that the non-healthcare participating, low level of social services, (and I might add non-unionized) deep south have the worst levels of child poverty. The southwest is close to the south in poverty.
Some midwest states and the northeast have the most provisions in place to support families and children (and I might add high levels of school spending and unionized teachers), leading to better conditions for child wellbeing, as well as a higher economic and educational standing.
This results in Mississippi having about 33% of its children living in poverty and Massachusetts with 2%. A pretty stunning difference.
The Foundation’s recommendations to improve the wellbeing of children include provisions that improve the conditions for parenting: tax credits, food stamps, job training, family friendly business practices, and a massive national infrastructure program to create jobs.
Much of this will require increased revenue streams at the state and national levels. And that’s what’s behind the curtain that the billionaire class behind Students Matter and the rest of the “schools suck industry” don’t want you to look at.
Replies
FloydThursby1941 9 years ago9 years ago
Gary, why do you ignore subgroups who find a way to excel academically while in poverty? Most of my family doesn't care much abut education, and a couple of them have found a good career anyways. Their kids have still not even gone to college. Income doesn't get grades. We have food stamps, public health care, public housing. California spends a lot on welfare. Studying gets grades. The … Read More
Gary, why do you ignore subgroups who find a way to excel academically while in poverty? Most of my family doesn’t care much abut education, and a couple of them have found a good career anyways. Their kids have still not even gone to college. Income doesn’t get grades. We have food stamps, public health care, public housing. California spends a lot on welfare. Studying gets grades. The only point about poverty I agree with is that sometimes kids don’t have a parent or anyone who can help them and they genuinely, morally try to study but reach an impasse. However, the average white kid in California studies 5.6 hours a week and spends over 40 watching TV, playing games and on social media/entertainment internet. If you’re in poverty and think you can make it to the middle or upper middle class this way you are dreaming. It is a choice, you decide whether to read or watch or play, but the wrong choice is usually made. Even in the middle class, kids study half as much as in Europe or Asia. It isn’t money, it’s moral decision-making. If schools taught the work ethic of the upper middle class during childhood, kids would do much better. Most say do an hour or so a day, it’ll all work out. Not only that, but behavior in the classroom is mediocre at best. If you can watch TV you can study. Neither one takes more money than the other. In fact on average TV costs more as school books and library books are available for free. These are the facts, and they are undisputed.
Ann 9 years ago9 years ago
For our purposes here the study says that 60% of Hispanic Latino children aren’t reading proficiently in 4th grade. Considering the rapid increase in Latino children in this state over the past decade we’re in for trouble…..
FloydThursby1941 9 years ago9 years ago
I agree Ann, that's why we can't accept the union stance that poverty excuses poor academic performance. If that's true, because these kids are in poverty, they will do bad in school, and because they do bad in school, they will be in poverty as adults, and therefore their children and their teachers will all be excused for their children doing poorly in school, and then their children, ad infinitum. If no one … Read More
I agree Ann, that’s why we can’t accept the union stance that poverty excuses poor academic performance. If that’s true, because these kids are in poverty, they will do bad in school, and because they do bad in school, they will be in poverty as adults, and therefore their children and their teachers will all be excused for their children doing poorly in school, and then their children, ad infinitum. If no one is responsible for breaking the cycle, teachers can’t be blamed, parents, no more can be expected of them, they’re poor, they’re working two jobs (hint they watch TV with their kids more time than they read to them or study with them, it isn’t that second job most of the time), then it will never change. We will have a caste system and a higher percentage in the lower caste each generation.
This is why we must stop using this excuse and hold up as sacrosanct the child raising methods of those who do raise children in poverty to excel in school anyways (Chinese, Koreans, Vietnamese, Indians, Cubans, Lebanese, Nigerians, Persians, Russians, Arabs, Polish, to name a few immigrant groups mentioned in ‘Triple Package’) and those teaching methods which push and reach children with low odds of success (KIPP, Harlem Zone, Geoffrey Canada’s approach, other successful charters which require after school and Saturday studying/tutoring). We need to break the cycle. If everyone poor should be expected to do poorly in school until we raise minimum wage to double the poverty line, forget it, we’re in for a declining standard of living in California. Teachers need to teach Latinos and everyone else how to organize their home life to prioritize school and excel academically).
Parent News Opinion 9 years ago9 years ago
Gary, how about if just 25% of the teacher evaluation was used in the review of test scores for teacher evaluations. Doesn’t that seem fair, and it may also satisfy those in the lawsuit who want test scores to be used for evaluations of teachers. Just 25% of test scores on the SmarterBalance/CAASPP to be used for teacher evaluation. 1/4 of the evaluation on a test that the teacher union has embraced fully.
Manuel 9 years ago9 years ago
This is silly. Why is it that every one assumes that the test scores are a valid measure of a child's learning? Have any of you realized that many students eventually realized that the tests don't matter (at least in California) because they don't count towards the grade they get in the class? That alone throws the results off so that they are useless to determine what students learned in a particular class. Test results are … Read More
This is silly.
Why is it that every one assumes that the test scores are a valid measure of a child’s learning?
Have any of you realized that many students eventually realized that the tests don’t matter (at least in California) because they don’t count towards the grade they get in the class? That alone throws the results off so that they are useless to determine what students learned in a particular class. Test results are only reliable when the student has a vested interest in getting the best score. Even then, the scores can be manipulated by specific training.
Sure, the test scores can be used to compare schools because, ON THE AVERAGE, all students will behave the same and the distribution of scores for a given school can be compared to another. But no way can that be said of a student to student. And it is even more difficult to use as a measure of a teacher’s effectiveness. Every single test designer has warned against this use at one time or another.
Would you believe that there are people who just can’t do well in tests no matter how well they try? We are not talking about the production of bolts here. We are talking about human beings with all their foibles and interests.
If you refuse to admit this, then you simply can’t accepts facts that have been proven over and over under many circumstances. Statistics can only go so far and that is what testing is all about: an exercise in statistics using as input the results of a questionably designed test that no one knows how it is constructed.
Using 25%, 30%, any percent is just an exercise in voodoo economics. And you all know how well that works.
FloydThursby1941 9 years ago9 years ago
Some kids don't try but teachers need to convince kids to try. Maybe there should be a dollar award to the best tests, $25 to the top score, $10 for the 2d, $5 for the 3d. This would motivate kids to push and try hard. But you have to ensure no cheating. Actually, it might be better, kids would quickly tell on a kid caught cheating. Tests are more accurate … Read More
Some kids don’t try but teachers need to convince kids to try. Maybe there should be a dollar award to the best tests, $25 to the top score, $10 for the 2d, $5 for the 3d. This would motivate kids to push and try hard. But you have to ensure no cheating. Actually, it might be better, kids would quickly tell on a kid caught cheating. Tests are more accurate than grades, which are biased by the difficulty of the school. Tests are the same for everyone regardless of race, religion, gender, etc. They are a morally neutral measure of human goodness. Some kids learn and study and others don’t. Those who work hard will be pluses to society and those who flake out and don’t pay attention or study will be minuses to society. It’s best to turn potential drags on society into pluses by teaching them how important this is.
Gary Ravani 9 years ago9 years ago
PNO: Here's a task for you. Do some homework on what the terms validity and reliability in tests actually mean. Hint: There are similarities to what is being talk about currently with political polls and "margin of error." All polls publish their margin of error as in + or - 3 points etc. Tests are very similar, but the terms are "standard error of measurement." And because there are two aspects, validity as well as reliability, … Read More
PNO:
Here’s a task for you. Do some homework on what the terms validity and reliability in tests actually mean. Hint: There are similarities to what is being talk about currently with political polls and “margin of error.” All polls publish their margin of error as in + or – 3 points etc. Tests are very similar, but the terms are “standard error of measurement.” And because there are two aspects, validity as well as reliability, it is more complex. You will find that, though test makers don’t talk about it much, that many tests don’t do well with either aspect. The old CA CSTs for example were very unreliable for students at either end of the performance scale. That would make them extraordinarily unreliable for use in teachers evaluations. Currently, because they are so new, we have no idea what the “standard error of measurement”, validity, or reliability of the Smarter Balanced tests may be. There is little reason to suspect that will be better than any other commercial test. That is the common problem with the use of test scores for evaluation. Because of the number of variables, differences in students and local conditions simply put, student test scores for evaluations are both invalid and unreliable. Other factors demonstrate that using test scores for accountability don’t add to student achievement and narrow the curriculum negatively affecting learning.
If someone said to you: “We know that X percent of a certain chemical added to your food will make you very ill. We promise to add only 25% of that amount of the chemical to your food. Is that acceptable?” What would your answer be?
FloydThursby1941 9 years ago9 years ago
Gary, we had a system before these tests where blacks were graduating high school 4 years behind whites in reading, 5 years behind Asians. Just letting every school and district and teacher decide what is best and hoping it works out is what we tried for a very long time, to horrible results. It doesn't have to narrow the curriculum. That is a sign of how idiotic kids get if they only … Read More
Gary, we had a system before these tests where blacks were graduating high school 4 years behind whites in reading, 5 years behind Asians. Just letting every school and district and teacher decide what is best and hoping it works out is what we tried for a very long time, to horrible results. It doesn’t have to narrow the curriculum. That is a sign of how idiotic kids get if they only study 5.6 hours a week or less in many cases. If you’re trying to teach kids to trick their way into looking better than they are, I guess that is a way of teaching to the test, but any kid who is taught well, does their homework and enthusiastically studies 3-5 hours a week and has parents who prepare them for Kindergarten and read to them will do well on the tests. We need to raise the bar on home support, parenting and effort. In Europe and Asia, 5.6 hours is a minimal effort and disreputable but here it is average. If you just pay attention and study you’ll do fine on the test. The reason we need so much monitoring is because when we did what you are suggesting, a lot of kids were spending a lot of time in school, very little studying outside of it and learning very little. When a kid can barely read after 12th grade and we spent $130k educating him, what an incredible waste, and that kid had 25000 hours of leisure time wasted over a childhood. I mean, what a waste of humanity. We can do better. Everyone should have skills.
Don 9 years ago9 years ago
Floyd, do a little digging yourself and you will find that VAM results do NOT consistently identify the same teachers as high or low performers. For example: “For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and classes that teachers teach. One study found that across five large urban districts, … Read More
Floyd, do a little digging yourself and you will find that VAM results do NOT consistently identify the same teachers as high or low performers.
For example:
“For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and classes that teachers teach. One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only predict from 4% to 16% of the variation in such ratings in the following year.”
Economic Policy Institute, “Problems with the Use of Student Test Scores to Evaluate Teachers”,
Replies
Ann 9 years ago9 years ago
https://www.aeaweb.org/articles.php?doi=10.1257/aer.104.9.2633 This study says VAM is predictive of future success.
Manuel 9 years ago9 years ago
Not everything that is published is true. Besides, there are other articles out there that refute the claims made by Chetty et al. And, no, I won't bother dragging those out because, frankly, it does get old and by now it is a waste of time. Plus there is that pesky article in the New York Times (a bastion of liberal bias to you, perhaps) published way back on March 6, 2011: http://www.nytimes.com/2011/03/07/education/07winerip.html?_r=0 that shows the Value-Added Model … Read More
Not everything that is published is true.
Besides, there are other articles out there that refute the claims made by Chetty et al. And, no, I won’t bother dragging those out because, frankly, it does get old and by now it is a waste of time.
Plus there is that pesky article in the New York Times (a bastion of liberal bias to you, perhaps) published way back on March 6, 2011:
http://www.nytimes.com/2011/03/07/education/07winerip.html?_r=0
that shows the Value-Added Model used by New York State then, followed by a horror story of what this formula did to a particular teacher rating from one year to the next. (The peanut gallery should note that the same type of arguments have surface with the imposition of the PARC assessments.)
Now, if you believe that the formula is a valid way of judging the effectiveness of ALL teachers according to VAM, then there’s no point in having any further conversation.
(For those not too bored by this, do note that all the coefficients in this formula are empirically derived from the data and many of them are “postulated” to begin with. Now, if you know what it takes to manipulate a matrix that big through a computer, you will know that this is a tall order. Specially since the coefficients will vary from year to year. It just a complicated model that probably has no bearing with reality because it is an exercise on correlations. And correlations are not causation.)
Ann 9 years ago9 years ago
Oh, here are the authors of your study from a genuine left of center organization. . "Problems with the use of student test scores to evaluate teachers" By Richard J. Shavelson, Robert L. Linn, Eva L. Baker, Helen F. Ladd, Linda Darling-Hammond, Lorrie A. Shepard, Paul E. Barton, Edward Haertel, Diane Ravitch, and Richard Rothstein | August 27, 2010 EPI is a 501(c)(3) corporation. In 2010 through 2012, a majority of our funding (about 60%) was … Read More
Oh, here are the authors of your study from a genuine left of center organization. .
“Problems with the use of student test scores to evaluate teachers”
By Richard J. Shavelson, Robert L. Linn, Eva L. Baker, Helen F. Ladd, Linda Darling-Hammond, Lorrie A. Shepard, Paul E. Barton, Edward Haertel, Diane Ravitch, and Richard Rothstein | August 27, 2010
EPI is a 501(c)(3) corporation. In 2010 through 2012, a majority of our funding (about 60%) was in the form of foundation grants, while another 26% came from labor unions. EPI also receives support from individuals, corporations, and other organizations
Don 9 years ago9 years ago
Ann, I more or less agree with your views in general and I am aware of the leanings of EPI. But as concerns VAM, I don't think it is ready for prime time as a form of teacher evaluation and I'm not sure that it could ever be given the shortcomings of the foundational data produced by standardized tests and the extreme concept of applying large scale statistical modeling to individuals. I'm not going to … Read More
Ann, I more or less agree with your views in general and I am aware of the leanings of EPI. But as concerns VAM, I don’t think it is ready for prime time as a form of teacher evaluation and I’m not sure that it could ever be given the shortcomings of the foundational data produced by standardized tests and the extreme concept of applying large scale statistical modeling to individuals. I’m not going to sanction VAM simply because I believe teachers ought to have more rigorous evaluations and less bulletproof job protections. Anchoring my views to solid science and reason precludes the use of VAM. It is all well and good if policy wonks want to use the data for large scale analysis of district or school progress towards reasonable goals, but on the micro level application firing teachers ought to be based upon human observations and interaction, not what the software spits out. Call me naive.
Don 9 years ago9 years ago
Whether VAM in its various incarnations is a useful tool for teacher evaluations is not the subject of this suit, but rather whether the defendants are violating the Stull Act. Nevertheless, it would be fairer to judge teachers using a contextual barometer like VAM compared to unfiltered, absolute test scores. Not to say I personally favor using VAM at all, flawed or not. The reason I oppose its use is because student test-based teacher … Read More
Whether VAM in its various incarnations is a useful tool for teacher evaluations is not the subject of this suit, but rather whether the defendants are violating the Stull Act. Nevertheless, it would be fairer to judge teachers using a contextual barometer like VAM compared to unfiltered, absolute test scores. Not to say I personally favor using VAM at all, flawed or not. The reason I oppose its use is because student test-based teacher accountability encourages teaching to the test and the standardization or homogenization of public education, that is, its devolution.
Replies
Todd Maddison 9 years ago9 years ago
Don, if you think standardization of learning in the basic subjects is a bad thing, you're part of the problem - not the solution. Certainly there are aspects of education that need to change based on the student or the region, but 2 + 2 is always 4 no matter where you are. Everything in education is not subjective. There are basic skills that all human beings need to succeed in life, and we need our … Read More
Don, if you think standardization of learning in the basic subjects is a bad thing, you’re part of the problem – not the solution.
Certainly there are aspects of education that need to change based on the student or the region, but 2 + 2 is always 4 no matter where you are.
Everything in education is not subjective. There are basic skills that all human beings need to succeed in life, and we need our schools to focus on those and do the best possible job of educating our children in those. You can call that “teaching to the test” in an inflammatory way if you want, but that’s just true.
That is how processes improve. You can see it all around you, but yet you’re incapable of seeing that in the education process?
If, for instance, you went to the DMV, one metric might be “the amount of time you spend in line”. Some DMV’s might score better on that than others. Just because a DMV gets you through the line quickly does not mean that the agent you speak with is intelligent, friendly, and helpful – but the metric of “average wait time” is very much a key factor that everyone who ever visits a DMV wants to be low.
Just because you can’t measure “agent effectiveness” once you get there doesn’t mean that measuring average wait time is invalid. It just means that the local DMV management needs to do their job in firing ineffective employees in ADDITION to managing their wait times.
I want a school system that can be guaranteed to provide all kids – everywhere – with the same basic level of competence in core subjects. From there of course school systems, teachers, etc are welcome to “do better than that” – and I expect the local administrations (driven by parents, of course) to push that part of it – but, again, simply because we want our math teachers to instill a LOVE of math in their students doesn’t mean we can’t also expect them to produce kids that know what 2 + 2 adds up to.
Don 9 years ago9 years ago
Todd, I think you're confusing the 10th Amendment and the rights of states and local communities to influence the public schools with changing the laws of physics or math - as in allowing incorrect answers (to use your example) in the name of individual freedom and local control. That is really a ridiculous reading of my comment. Or perhaps I was a little too brief, opening up my comment to your interpretation. The problems in education … Read More
Todd, I think you’re confusing the 10th Amendment and the rights of states and local communities to influence the public schools with changing the laws of physics or math – as in allowing incorrect answers (to use your example) in the name of individual freedom and local control. That is really a ridiculous reading of my comment. Or perhaps I was a little too brief, opening up my comment to your interpretation.
The problems in education are many and standards rate low when it comes to achievement drivers such as family influence, culture, income, class size and teacher quality, in no particular order. Academic standards have a last place showing in comparison and, therefore, I would base my reform of education and the future of teachers on a poor bet and a test, the latter of which has been foisted upon the state without a proper researched-based vetting.
I’m all for evaluating teacher quality and certainly some kinds of commonly accepted measures need to be employed in doing so. But you seem to be saying that education needs to be leveled so we can measure teacher quality while I’d say it is the other way around – teacher quality needs to be leveled so education quality is leveled.
Don 9 years ago9 years ago
Sorry ” I would NOT base…”
FloydThursby1941 9 years ago9 years ago
What about effort being leveled?
Manuel 9 years ago9 years ago
Mr. Maddison, your comments make it clear that you believe that "metrics" are the basis for improvement of any "process." Indeed, one needs to "know" the problems in order to change the outcome of a "process." But it doesn't follow that the "metrics" will help you in changing the outcome in a specific direction. Yes, imparting previous knowledge ("2 + 2 = 4") is one part of the educating process. Helping a student to know what … Read More
Mr. Maddison, your comments make it clear that you believe that “metrics” are the basis for improvement of any “process.”
Indeed, one needs to “know” the problems in order to change the outcome of a “process.” But it doesn’t follow that the “metrics” will help you in changing the outcome in a specific direction.
Yes, imparting previous knowledge (“2 + 2 = 4”) is one part of the educating process. Helping a student to know what to do with previous knowledge is another (“4 + 4 = “(2 + 2) + (2 + 2) = 8”). Yet another part is where else can that knowledge be applied. “Metrics” might be able to help with the first two instances, but it won’t on the third. If improvement of the “process” depends on something that can’t truly be measured, then “metrics” won’t be of any help at all.
You seem to be believe that standardized testing is the only way to go.
Are you at all familiar with how this particular sausage is made?
And if you are, are you aware that you can always train anyone to pass any given standardized test (and let’s not even get into the influence of the test maker’s “culture” on the final product in regards to its application to those who don’t share the test maker’s “culture”)? What then is the point of such tests, specially since they don’t tell us anything on the ability of a test taker to apply established knowledge to new situations? And then on top of that claim that a classroom teacher is the major influence on a test taker’s score?
If you know all this and you still insist that this is the only way to go and let “the invisible hand of the market” decide, then you learned nothing from the failure of the quants to maximize profits while pretending to predict human behavior.
Mathematical models are just that: models. They are not “The Truth” and will always be approximations. Some of them might approximate reality, but many, specially those not supported by data, will crash and burn spectacularly. Stack-and-rank has already proven a failure at MicroSoft. VAM, which is a meandering offshoot of Stack-and-rank is also a failure when applied to determining teacher effectiveness as the New York mess attests.
I just can’t understand why people like you pretend that mathematical models justify your political and/or ideological position. Why prostitute mathematics? Why not just admit that you are a believer in the Bell Curve and all that it implies and be done with it?
As for the Stull Act, it is what it is because Villaraigosa help place a booby trap in it when he, in 1999 and as a member of the State Legislature, snuck in the language that has led to this mess. Even then, the actual language of the Stull Act (Ed Code 44660-44665, I think), allows a lot of wiggle room that Welch et al don’t want to deal with. In particular, the tests used for evaluating students must be “criterion referenced assessments” and the CSTs and neither the SBACs seem to be that. Plus that pesky part about negotiating those “assessment guidelines” with the teachers (Ed Code 44661-44662). But, hey, they can afford the lawyers and the time for arguing in court about this.
Ann 9 years ago9 years ago
Wow, Please share which school of education you attended. It sounds like you were fed the same line as I. Everyone else who haven't had the intellectually painful honor of attending, this is where I believe the trouble begins. If we could rid ourselves of "schools of education", at least the current model, we be a long way down the road to having better teachers. Just as an aside the CST was absolutely a … Read More
Wow, Please share which school of education you attended. It sounds like you were fed the same line as I. Everyone else who haven’t had the intellectually painful honor of attending, this is where I believe the trouble begins. If we could rid ourselves of “schools of education”, at least the current model, we be a long way down the road to having better teachers. Just as an aside the CST was absolutely a criterion referenced test.
Manuel 9 years ago9 years ago
The CST was a criterion referenced test? And what do you base such a belief? Simply that they told you it was? Ever looked at the distribution of scores? They never changed significantly from year to year. If it was a criterion test, teachers would have figured out how to teach to the test! Besides, the CDE itself admits in p. 6 of this document http://www.cde.ca.gov/ta/tg/ai/documents/keyelements0504.pdf that CST items were evaluated based on tryouts and field tests and … Read More
The CST was a criterion referenced test? And what do you base such a belief? Simply that they told you it was?
Ever looked at the distribution of scores? They never changed significantly from year to year. If it was a criterion test, teachers would have figured out how to teach to the test!
Besides, the CDE itself admits in p. 6 of this document
http://www.cde.ca.gov/ta/tg/ai/documents/keyelements0504.pdf
that CST items were evaluated based on tryouts and field tests and then selected on the basis of their p-value. A criterion-reference test would never be concerned about p-values but only whether students learn specific information. Try again.
As for “learning processes,” I hate to burst your bubble but this is based on my own experience with highly gifted children. And maybe I’ll tell you my school when you tell me yours.
navigio 9 years ago9 years ago
Even though VAM may be better than raw (is it even that?), it may still be worse than not using test scores in any manner. I expect this is why the stull act is mostly ignored. I doubt this is a question the lawsuit will bother with even though its the one that really should be answered.
Todd Maddison 9 years ago9 years ago
One of the more unfortunate things about this kind of comment trail is how quickly it exposes the basic lack of understanding amongst educators of how processes are improved. In any process you need to establish metrics by which you can tell if you've been successful or not. If you make a change in the process and it results in poorer results, that change was not good. If you make a change … Read More
One of the more unfortunate things about this kind of comment trail is how quickly it exposes the basic lack of understanding amongst educators of how processes are improved.
In any process you need to establish metrics by which you can tell if you’ve been successful or not. If you make a change in the process and it results in poorer results, that change was not good. If you make a change and results are better, that change is something that should then be implemented throughout the systems.
Without setting up any form of metrics to measure your results, you will never – ever – improve any process.
I realize that teachers do not have the expertise to determine that – it’s not their core competency – but there most certainly are those who do have those skills.
Those who post airy-fairy things like “teachers should be peer reviewed honestly” are just exposing their lack of understanding of how process improvement works.
If you’re running a retail chain and you ask the staff of a specific store to peer review their fellow workers, they may feel many of those workers are excelling – because they have no clue that a store a mile away is doing double the sales or profit.
The starting point for any effective evaluation process is to set those metrics – which can be done accounting for variables like economic status, prior academic performance, etc. Anyone who says that simply can’t be done is – again – someone who has never depended on “making their company more effective” for their paycheck.
If teachers don’t want to accept standardized tests as that method, I’d be fine going to a voucher system where I can take that money and apply it to “any school I want to”. Parents will then make the decision on whether a school is performing adequately or not, and those that lose enrollment will get closed – meaning those teachers who did not want to be evaluated will then have no jobs.
If they think that’s preferable, I’m down with it. What I’m not down with is letting the employees design the system by which their performance is judged – that’s managements job, and one that is usually abdicated in the educational world….
Replies
Gary Ravani 9 years ago9 years ago
Todd: The problem is quantitative aspects of education overtook the qualitative aspects. One of the key reasons for this was higher quality assessments are expensive and time consuming. Reducing education to "metrics" was cheap, convenient, and useful to those who wanted to take a quick hit at public education (called' the schools sucks industry" by one revered commentator). As familiar as you seem to be with AN "improvement process" it seems one more appropriate to industrial … Read More
Todd:
The problem is quantitative aspects of education overtook the qualitative aspects. One of the key reasons for this was higher quality assessments are expensive and time consuming. Reducing education to “metrics” was cheap, convenient, and useful to those who wanted to take a quick hit at public education (called’ the schools sucks industry” by one revered commentator).
As familiar as you seem to be with AN “improvement process” it seems one more appropriate to industrial based activities and not human based one focused around relationships.
It’s an easy mistake to make. Read about the motivations of those who put NCLB and the focus on metrics over kids and learning. It was about embarrassing schools and promoting an agenda to push vouchers and charters. If you poke around the Ed Week site you can find a number of articles that substantiate those assertions.
Good luck.
Todd Maddison 9 years ago9 years ago
A common response - that somehow it's impossible to measure educator success because of "the human factors" involved. As if that means there's no possible way to do it. Certainly there are both quantitative and qualitative measures that are important. In business, we might have one measure that says "what is your stores' gross profit" - which can be measured daily from the accounting systems - but then another that might focus on "customer … Read More
A common response – that somehow it’s impossible to measure educator success because of “the human factors” involved. As if that means there’s no possible way to do it.
Certainly there are both quantitative and qualitative measures that are important. In business, we might have one measure that says “what is your stores’ gross profit” – which can be measured daily from the accounting systems – but then another that might focus on “customer satisfaction”, often taken from surveys.
Of course those who score poorly in the latter often argue that there are unmeasurable factors involved – “only the unhappy customers respond”, etc. Of course that is also true – but that doesn’t mean that there aren’t ways to compensate for those factors in the results.
That also doesn’t mean that there isn’t a component of any review process that boils down to “does the boss think I’m doing a good job?” Humans are necessary in the loop, that’s why we hire management.
In the schools, there are very much quantitative measures possible. Standardized tests are very obviously the best way to do that. Should those results be the ONLY metrics? Of course not, but they still need to be a big part of the final results.
If we saw results that said (say), 4 out of 5 teachers in a particular grade/subject were improving their classes test scores by X% from beginning to end of school year, but one teacher did not, it would then be up to the principal to make their determination as to why. That’s what management is for.
As a parent, if that principal gave that teacher a “pass” for a year or two – maybe they got a particularly disruptive set of students or something – I’m fine with that, but if that teacher consistently fails to meet the same standards all other peer teachers meet, then that teacher is simply not as good as the rest. Again, it’s management’s job to step in.
From there, the District should be monitoring that entire schools’ performance, to make sure the principal is indeed doing their job. And so on up the chain. Again, that’s managements’ job.
What we see in education is often this attitude where we build a complete fantasy land where no measurement is considered valid, so “everyone involved is above average”. That’s patently not true, and is why our schools often fail to provide the results we expect.
“Inspect what you expect” is how the real world improves results, we need that in the schools as well.
Gary Ravani 9 years ago9 years ago
Todd; You continue to display your lack of background in education research or policy. Your statement: "Standardized tests are very obviously the best way to do that.." exemplify that lack of knowledge. There is a plethora of solid research refuting your assertion about standardized test scores. The easiest to access is the report done by the National Research Council showing that a decade of using such test scores to drive policy changes, including accountability for teachers, … Read More
Todd;
You continue to display your lack of background in education research or policy. Your statement: “Standardized tests are very obviously the best way to do that..” exemplify that lack of knowledge. There is a plethora of solid research refuting your assertion about standardized test scores. The easiest to access is the report done by the National Research Council showing that a decade of using such test scores to drive policy changes, including accountability for teachers, has not resulted in improved achievement and it has harmed learning overall by narrowing the curriculum. Numerous other studies have shown that student tests scores are far too unreliable and/or invalid ( due to the number of variable involved) to use to make high stakes decisions about students, let alone teachers.
You may want to move over to a forum at Bloomberg news or Business Week or somewhere.
Don 9 years ago9 years ago
Intent on the idea of measuring performance, I suspect Todd welcomes whatever metric he can get hold of as a valid evaluative tool. But anyone so intent upon rating, sorting and stacking ought to put great stock in accuracy, otherwise the whole approach is a house of cards.
FloydThursby1941 9 years ago9 years ago
That's why Principals would have to dig deeper. Todd is right, one bad year may be explainable, but if you are behind peers year after year, a principal needs to figure out why. Pressure to improve is impossible if there is no measurement or no consequences for measurements. You're right Don that it has to be accurate, but the status quo seems to say this may be accurate, implying they have a … Read More
That’s why Principals would have to dig deeper. Todd is right, one bad year may be explainable, but if you are behind peers year after year, a principal needs to figure out why. Pressure to improve is impossible if there is no measurement or no consequences for measurements. You’re right Don that it has to be accurate, but the status quo seems to say this may be accurate, implying they have a better solution. However, their solution is always continue to judge by seniority. I’ve never heard Gary or someone like him point out a reason why a metric lacks accuracy, then propose a solution to make it more accurate. I think if we are all trying to create an accurate measure, we can do so, but the issue is many don’t want any universal measure.
Don 9 years ago9 years ago
Do you know anything about VAM, Floyd and Todd? Read John Ewing's article in the WaPo Answer Sheet for a brief but succinct explanation. http://www.washingtonpost.com/blogs/answer-sheet/post/leading-mathematician-debunks-value-added/2011/05/08/AFb999UG_blog.html Here are some excerpts with the concluding 2 paragraphs first, but you, Floyd and Todd need to read the whole article or any number of others that have explained the values and limitations of VAM in the face of widespread and popular misunderstanding of the utility of the model: "When we accept … Read More
Do you know anything about VAM, Floyd and Todd?
Read John Ewing’s article in the WaPo Answer Sheet for a brief but succinct explanation.
http://www.washingtonpost.com/blogs/answer-sheet/post/leading-mathematician-debunks-value-added/2011/05/08/AFb999UG_blog.html
Here are some excerpts with the concluding 2 paragraphs first, but you, Floyd and Todd need to read the whole article or any number of others that have explained the values and limitations of VAM in the face of widespread and popular misunderstanding of the utility of the model:
“When we accept value-added as an “imperfect” substitute for all these things because it is conveniently at hand, we are not raising our expectations of teachers, we are lowering them. And if we drive away the best teachers by using a flawed process, are we really putting our students first?
Whether naïfs or experts, mathematicians need to confront people who misuse their subject to intimidate others into accepting conclusions simply because they are based on some mathematics. Unlike many policy makers, mathematicians are not bamboozled by the theory behind VAM, and they need to speak out forcefully. Mathematical models have limitations. They do not by themselves convey authority for their conclusions. They are tools, not magic. And using the mathematics to intimidate — to preempt debate about the goals of education and measures of success — is harmful not only to education but to mathematics itself.
Yet most of those promoting value-added modeling are ill-equipped to judge either its effectiveness or its limitations. Some of those who are equipped make extravagant claims without much detail, reassuring us that someone has checked into our concerns and we shouldn’t worry. Value-added modeling is promoted because it has the right pedigree — because it is based on “sophisticated mathematics.”As a consequence, mathematics that ought to be used to illuminate ends up being used to intimidate. When that happens, mathematicians have a responsibility to speak out.
This last problem plays a larger role as the stakes increase. This is often referred to as Campbell’s Law: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to measure” [Campbell 1976]. In its simplest form, this can mean that high-stakes tests are likely to induce some people (students, teachers, or administrators) to cheat … and they do [Gabriel 2010].
But the more common consequence of Campbell’s Law is a distortion of the education experience, ignoring things that are not tested (for example, student engagement and attitude) and concentrating on precisely those things that are.
• In the real world of schools, data is frequently missing or corrupt. What if students are missing past test data? What if past data was recorded incorrectly (not rare in schools)? What if students transferred into the school from outside the system?
• The modern classroom is more variable than people imagine. What if students are team-taught? How do you apportion credit or blame among various teachers? Do teachers in one class (say mathematics) affect the learning in another (say science)?
• Every mathematical model in sociology has to make rules, and they sometimes seem arbitrary. For example, what if students move into a class during the year? (Rule: Include them if they are in class for 150 or more days.) What if we only have a couple years of test data, or possibly more than five years? (Rule: The range three to five years is fixed for all models.) What’s the rationale for these kinds of rules?
• Class sizes differ in modern schools, and the nature of the model means there will be more variability for small classes. (Think of a class of one student.) Adjusting for this will necessarily drive teacher effects for small classes toward the mean. How does one adjust sensibly?
• While the basic idea underlying value-added models is the same, there are in fact many models. Do different models applied to the same data sets produce the same results? Are value-added models “robust”?
•Since models are applied to longitudinal data sequentially, it is essential to ask whether the results are consistent year to year. Are the computed teacher effects comparable over successive years for individual teachers? Are value-added models “consistent”?
These last two points were raised in a research paper [Lockwood 2007] and a recent policy brief from the Economic Policy Institute, “Problems with the Use of Student Test Scores to Evaluate Teachers”, which summarizes many of the open questions about VAM:
“For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and classes that teachers teach. One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only predict from 4% to 16% of the variation in such ratings in the following year.
“Thus, a teacher who appears to be very ineffective in one year might have a dramatically different result the following year. The same dramatic fluctuations were found for teachers ranked at the bottom in the first year of analysis. This runs counter to most people’s notions that the true quality of a teacher is likely to change very little over time and raises questions about whether what is measured is largely a “teacher effect” or the effect of a wide variety of other factors [Baker 2010, 1].”
TheMorrigan 9 years ago9 years ago
Five items I found very troubling about VAM (a principal's anecdotal perspective): 1) When the same teacher teaches two separate grade levels and is labeled highly effective for 8th grade and mostly ineffective for 7th grade, it might give you pause. Then when the same teacher the next year gets mostly ineffective for 8th grade and highly effective in 7th, it might make you scratch your head more. Although it is a head scratcher, this might … Read More
Five items I found very troubling about VAM (a principal’s anecdotal perspective):
1) When the same teacher teaches two separate grade levels and is labeled highly effective for 8th grade and mostly ineffective for 7th grade, it might give you pause. Then when the same teacher the next year gets mostly ineffective for 8th grade and highly effective in 7th, it might make you scratch your head more. Although it is a head scratcher, this might give you some perspective: The same kids moved from year to year with the same teacher.
2) When we switched VAM models, it turned my staff upside down. The ones who were labeled highly effective for the last three years in a row were now label marginal or mostly ineffective. The ones who were labeled marginal or mostly ineffective for the last two years in a row were labeled effective or highly effective. I had minus and plus points all over the place. It was a nightmare.
3) Like Chetty and his brothers in arms, I found that my math teachers’ VAM scores mostly aligned with my own principal observations. There were a few outliers, but for the most part, VAM tended to be somewhat stable here. And if that were the case I could live with it. However, like Chetty and his buddies also show but it somehow gets very little press, I found that my ELA teachers’ VAM scores did not mostly align with my observations. I found that ELA teachers have VAM schools that are as stable as a coin toss.
4) I always went with the motto that a principal’s job was to help teachers become better teachers. However, I found that VAM did nothing to help me help them. To be honest, I found VAM to be a blunt and ineffective tool that sometimes gave reliable information but never helped me help my staff. I found it to be a cumbersome and crude tool for me; but for an ineffective administrator who doesn’t care about helping teachers improve, I suspect that VAM makes it so much easier to say “It says that” or “It shows that” without actually investing any time into trying to help the teacher improve while simultaneously sounding authoritative and expert-like.
5) Try explaining the VAM rationale of it all to a teacher who you know is superb and highly effective but who got marginal or mostly ineffective one year. Trust me, you will sound like a moron and an a**hat.
Manuel 9 years ago9 years ago
@TheMorrigan Your fifth point is particularly correct because no one told you how the score is arrived at. As I pointed elsewhere, once the formula is published and the amount of number massaging that needs to take place is explained, people will realize this is moronic because it will certainly lead to variations such as those you observed. In case you forgot, the formula used in New York was published back in 2011 by the Gray … Read More
@TheMorrigan
Your fifth point is particularly correct because no one told you how the score is arrived at.
As I pointed elsewhere, once the formula is published and the amount of number massaging that needs to take place is explained, people will realize this is moronic because it will certainly lead to variations such as those you observed.
In case you forgot, the formula used in New York was published back in 2011 by the Gray Lady:
http://www.nytimes.com/2011/03/07/education/07winerip.html?_r=0
A variation of that formula can be found in “technical webinar” that LAUSD made available to describe “Academic Growth over Time”, Deasy’s method (actually shepherded by two TFAers who became high-level LAUSD educrats). Too bad it is no longer there, probably because the contract with the Value-Added Research Center (based at the University of Wisconsin-Madison) expired.
The bottom line is that VAM will never be ready for prime time, regardless of the missionary zeal of its proponents.
Ann 9 years ago9 years ago
Oh wow! Sorry, Todd no alternative views allowed by Heil Ravini! Well good thing you and your fellows don’t run this blog and hopefully the likes of your group of edspeakers of mostly nonsense won’t survive Vergara and the latest court challenge on evaluations! Of course schools can be measured for success and while a pleasant climate, good attendance etc. are great to have, academic success is their mandate and teachers are responsible for making that happen.
John Fensterwald 9 years ago9 years ago
Ann et al: I have enjoyed the rich conversation and diverse perspectives that the piece has generated. Let me remind you and others -- especially new commenters to the site -- to read our comments policy and to maintain civility. No personal aspersions, like "Heil Ravani," and attacks on other commenters, including speculation of motives of people you don't know. The alternative is place you and others in moderation or to suspend the privilege of commenting … Read More
Ann et al:
I have enjoyed the rich conversation and diverse perspectives that the piece has generated. Let me remind you and others — especially new commenters to the site — to read our comments policy and to maintain civility. No personal aspersions, like “Heil Ravani,” and attacks on other commenters, including speculation of motives of people you don’t know.
The alternative is place you and others in moderation or to suspend the privilege of commenting — an option we’d prefer not to do.
Now, back to the discussion.
Gary Ravani 9 years ago9 years ago
Thanks for the thought, John. However it appears that "Ann," along with several others of the anonymous keyboard commentators who get the vapors when I write (based on 35 years of classroom experience along with 25 years as a union state officer with considerable experience working on policy issues in Sacramento) needs to go on my persona non grata list. I will not be directly responding to her any longer saving us all some aggravation. BTW, … Read More
Thanks for the thought, John. However it appears that “Ann,” along with several others of the anonymous keyboard commentators who get the vapors when I write (based on 35 years of classroom experience along with 25 years as a union state officer with considerable experience working on policy issues in Sacramento) needs to go on my persona non grata list. I will not be directly responding to her any longer saving us all some aggravation.
BTW, Heil Ravani. Really? Odd reference to someone who writes distinctly from the “left” side of the aisle. Not the oddest though.
Jordan B Smith Jr. (EdD/CI) 9 years ago9 years ago
It is obvious that Theodore Boutrous and Student Matter legal representativd have never taught secondary mathematics in a public school under NCLB Act of 2001 and have never been the subject of an workplace evaluation without accountability. I became a teacher by taking a test and by earning my degree which was also subject to taking a test. I was responsible for my score. I had the choice to prepare for the test … Read More
It is obvious that Theodore Boutrous and Student Matter legal representativd have never taught secondary mathematics in a public school under NCLB Act of 2001 and have never been the subject of an workplace evaluation without accountability. I became a teacher by taking a test and by earning my degree which was also subject to taking a test. I was responsible for my score. I had the choice to prepare for the test because the results would count against me and only me if I failed the test. In college if you fail a course you will have to pay for that course again. There is no “free repeating.”
In our public school system a student can fail every math test and still move forward in the system through the 8th grade. It is not until the 9th grade that reality sets into a student’s life when they find out that just coming to school will not be sufficient to graduate from high school. You have to pass the course to earn credits. So, there is personal responsibility for the student. The student is fully aware of the connection of credits to grades. He or she is personally responsible for passing the CAHSEE in high school. If students don’t pass they don’t get a diploma. In fact, the student has eight opportunities to take the test based on 7th grade math standards, Algebra I, and Geometry.
But there is no student accountability for any of the state tests. In fact the results of these standardized test are not received until months after school has ended. Students know that the test results are not included in their grades. If a test is not connected to earning credits or other personal rewards then there is no incentive for the student to perform well on the test. I have had 4.0 students who have simply bubbled in the state test scoring below basic because it did not count in their GPA. So as a teacher I ask why these students scores should be part of my evaluation? It’s not my test!.
The new Common Core mathematics assessment field testing of the SBAC and The California Assessment of Student Performance and Progress (CAASPP) System performance tasks to 11th graders in 2014 and 2015 witnessed students simply writing “I don’t know” or “Who cares” when the student is aware that the results will not impact their grade. Again, its not my test!
Teacher evaluations should be about the proper use for formative and summative assessments. You can evaluate a teacher about how , when, where,and why they should use formative assessments. Teachers who do not use formative assessments should not be teaching and should receive a poor evaluation. Teacher should know why and how to use summative assesssments. The key question for teacher evaluations is what does the teacher do with the information from the assessment? What does the teacher do when a student fails a test? What does the teacher do when a student is failing the class? How does the teacher modify instruction to enhance student learning?
Teachers will never object over an evaluation of their instructional methods to enhance student learning. If you want to evaluate my teaching using a test then I will be the one taking the test! Would you fly to San Francisco in a plane with an assigned pilot with no flight hours? Why do lawyers, judges, and politicians try to fly in classroom of a public school without a credential or teaching experience?
Replies
John Fensterwald 9 years ago9 years ago
Jordan: Starting next year, teachers should receive Smarter Balanced test results within four weeks of completing their teachers tests. They'll have access to a sophisticated data system that will give them the ability to analyze test results in detail, pointing out, say, how sixth grade students did with dividing by fractions compared with other content they learned during the year. Teachers and their evaluations can use this information to help identify strengths and weaknesses in … Read More
Jordan: Starting next year, teachers should receive Smarter Balanced test results within four weeks of completing their teachers tests. They’ll have access to a sophisticated data system that will give them the ability to analyze test results in detail, pointing out, say, how sixth grade students did with dividing by fractions compared with other content they learned during the year. Teachers and their evaluations can use this information to help identify strengths and weaknesses in instruction, which, after all, is the crux of an evaluation — as well as the goal of continued improvement. I heard an overview of the system during a demonstration to the State Board this month and hope to get a better look sometime soon.
It was unfortunate that all of the protocols for teacher access to test results couldn’t be worked out by now. However, teachers should be getting the information starting in August. I don’t know if the same level of information will be available with interim Smarter Balanced tests.
On another matter, just to repeat: The Stull Act is silent regarding the extent to which results on state tests — or any tests — should factor into an evaluation; it certainly doesn’t mention VAM. A victory by Students Matter in the lawsuit wouldn’t change the law. Districts– at least those in the lawsuit – would be required to follow it and make test results and other forms of measurable student achievement at a minimum part of the conversation in an evaluation.
Parent News Opinion 9 years ago9 years ago
Dear Mr. Fensterwald, What about the parents. Can you check to see if the parents will get the same fast results of how their children scored on the SmarterBalance/CAASPP test by mail, or to save monies, maybe, can you suggest that mail not be used and all parents must instead check the scores with possible print out, by computer, on line, and if any low income parents do not have access, to the computer, … Read More
Dear Mr. Fensterwald,
What about the parents. Can you check to see if the parents will get the same fast results of how their children scored on the SmarterBalance/CAASPP test by mail, or to save monies, maybe, can you suggest that mail not be used and all parents must instead check the scores with possible print out, by computer, on line, and if any low income parents do not have access, to the computer, then they can, at any school put a sort of simple 1/3 page cut slip to request a print out of the SmarterBalanceCAASPP test results for their child. and or even a nice idea would be for all teachers to be given the time and responsibility to staple the print out of the SmarterBalance/CAASPP test to the report card they give to the parents in the child’s backpack.
Why are we spending so much for vast mail outs of paper and such when we can save monies by using on line and offering low income parents the choice of a print out. There can be an opt in for on line results and or a choice for opt out for the school to hand print the SmarterBalanceCAASPP test results and staple it to all report cards each year.
We need to save monies and put those monies into the hiring of additional resources for low income low preforming children and also, some non low income children are low preforming too, so Mr. Fensterwald, can you suggest that to the powers that be whom you sometimes brush shoulders with, and can you do a quick cost saving analysis of what the California Department of Education spends to mail out all the SmarterBalance/CAASPP letters to all parents this year?
thanks,
Parent News Opinion
John Fensterwald 9 years ago9 years ago
Dear Parent: I can inquire regarding the use of email as opposed to snail mail. Perhaps parents should have the option of choosing which form they want; those that prefer the mail or don't have an email address would get a paper form. This year, the state is being cautious about releasing the results, waiting until the end of August so that teachers and principals can provide answers to parents' questions. There will be no comparisons on … Read More
Dear Parent:
I can inquire regarding the use of email as opposed to snail mail. Perhaps parents should have the option of choosing which form they want; those that prefer the mail or don’t have an email address would get a paper form.
This year, the state is being cautious about releasing the results, waiting until the end of August so that teachers and principals can provide answers to parents’ questions. There will be no comparisons on the results that parents receive this year — how a child’s 4-digit scale score compares with others in the state or district. It’s an open question for next year. Since state, district and school data will be available, why not provide information on the reports home. However, that would delay the results, since the state won’t post statewide data until all districts have completed testing, and the results have been checked and verified.
Teachers will get results of their students’ scores much earlier next year — within four weeks of the end of testing. The earlier release could be useful to them and their schools for figuring out over the summer how to improve instruction. If the math scores are used as a factor for placement purposes (who gets into Algebra I), parents can make the case that they should see the scores earlier, too.
Parent News Opinion 9 years ago9 years ago
Thank you Mr. Fensterwald for your clear well thought out reply, parents such as myself appreciate the neutral unbiased nature of your reporting and how you drive information in the middle of the fairway so to speak, free of political leanings, open to opposite views of discourse, you rarely seem to duck hook information, take a mulligan nor slice with biased views. Now, us parents, and I speak to many as well as attend educational meetings … Read More
Thank you Mr. Fensterwald for your clear well thought out reply, parents such as myself appreciate the neutral unbiased nature of your reporting and how you drive information in the middle of the fairway so to speak, free of political leanings, open to opposite views of discourse, you rarely seem to duck hook information, take a mulligan nor slice with biased views.
Now, us parents, and I speak to many as well as attend educational meetings open to public, us parents have been fed often side speak, misspeaks, and lots of silence combined with in my opinion a lack of change in schools.
We parents have had our words often heard but not listened to deeply enough to witness much changes to the status quo.
The low income parents are often very savvy with computers and the parents might wish to opt out of the $$$$ of the mailing of yearly SmarterBalance/CAASPP test results. Once read the multi colored graphics on paper are thrown away, so let us find out if Mr. Torlakson and the SBE want to save $$$$ and let parents opt if they wish for on line results as schools, districts and teachers do.
Now, I as a parent want the arrival of the SmarterBalance/CAASPP results to be shipped to me the same day as the teacher or school district gets the results. I do not want a time buffer.
Let the teachers and school districts and CDE and SBE show honest transparency in simultaneously releasing all information to all groups rather than manipulate the time releases. This will build trust for all stakeholders.
You state, “this year the state is being cautious about releasing the results…,”…I as a parent am losing trust in this experiment called SmarterBalanceCAASPPcombined with new math, a , in my opinion, lack of time for teachers to properly cover all mandated state standards in all subjects, and, I want PARENT VOICE to be given honor and sincere respect , I appreciate your comment back,
I do not think things are going too well with current oversight by the CDE and I believe the P.T.A. will need to be involved more by the C.D.E. and parents such as myself will need to craft out new demands in how children are educated for big wigs trying to navigate by use of a northern star may wish to borrow a GPS system parents use so to speak, with regard to us knowing when the current state of education is or may be in bad hands due to hippy like wants and needs combined with esoteric high paid think tank people who make up new hard to understand words to in some way try to bamboozle parents into a new educational experiment that is, in my opinion in serious need of outside overview by….
parents.
Parent Opinion News
Paul 9 years ago9 years ago
As a former teacher with middle-, high- and adult school math experience, I couldn't have said it better myself, Jordan. As someone who also has credentials in subjects for which there are no state standardized tests, I am perplexed that the work I did in a French, Spanish or business classroom should have been evaluated on the basis of students' state standardized test scores in other subjects. (Of course I linked ELA to all of my … Read More
As a former teacher with middle-, high- and adult school math experience, I couldn’t have said it better myself, Jordan.
As someone who also has credentials in subjects for which there are no state standardized tests, I am perplexed that the work I did in a French, Spanish or business classroom should have been evaluated on the basis of students’ state standardized test scores in other subjects. (Of course I linked ELA to all of my math, world language, and business teaching, but nobody cares about that.)
My credentials expire in 2017, and at 40, it is unlikely that I will have children of my own. I’m grateful to be able to watch from the sidelines as LawyersFirst fights to extend Wal-Mart human resource practices to public schools.
These “public interest” lawyers will be satisfied when all teachers (not just charter school teachers and teachers in their first 2+ years with a given school district) become at-will employees, subject to dismissal without cause. Why bother with evaluations at all? Just get a new teacher!
Parent News Opinion 9 years ago9 years ago
Jordan B Smith,,this comment for you.
See California senate bill 172.
See EdSource recent article on possible removal of high school exit exam called CASEE.
If passed, it would mean you are wrong in your statement that students now could just show up and earn a high school diploma, just keep the seat warm. What do you think about that?
Jerry Heverly 9 years ago9 years ago
I want to believe that organizations like Students Matter have a sincere interest in improving education (though, truthfully, I really think their intent is to undermine public education and promote private education) but just about every objective analysis I've seen on VAM proves to me, at least, that they are just--to quote Mr. Scalia--jiggery pokery. The algorithms used are full of debatable assumptions. The wide fluctuations for individual teachers from year to year suggests that the measurements … Read More
I want to believe that organizations like Students Matter have a sincere interest in improving education (though, truthfully, I really think their intent is to undermine public education and promote private education) but just about every objective analysis I’ve seen on VAM proves to me, at least, that they are just–to quote Mr. Scalia–jiggery pokery.
The algorithms used are full of debatable assumptions.
The wide fluctuations for individual teachers from year to year suggests that the measurements are fundamentally flawed.
The fact that teachers are often evaluated based on scores for students they have never even seen (especially music, art, and PE teachers) makes the whole process a mockery.
And the assumption that firing low performing teachers (if such people could credibly be identified) will improve education is also very debatable.
Using VAM just isn’t going to improve education. I wish Students Matter could have found another cause to spend their money on, something that might actually make a positive difference.
Replies
Tom 9 years ago9 years ago
"And the assumption that firing low performing teachers (if such people could credibly be identified) will improve education is also very debatable." ARE YOU KIDDING ME? Do you have young kids in school, or maybe used to? If so, apparently you have never experienced the negative effect a bad teacher has on young kids. They exist and there is ample evidence that many of them continue year after year … Read More
“And the assumption that firing low performing teachers (if such people could credibly be identified) will improve education is also very debatable.” ARE YOU KIDDING ME? Do you have young kids in school, or maybe used to? If so, apparently you have never experienced the negative effect a bad teacher has on young kids. They exist and there is ample evidence that many of them continue year after year in the classroom harming kids.
Deborah 9 years ago9 years ago
student test scores are exactly that - the student's. A different metric is needed or a variety of metrics. The formula to create test results varies year to year and is not relevant. Student acuity level should also matter just like doctors who treat more complex patients. Teachers should peer review themselves and actually evaluate each other honestly not just for their own benefit or out of fear. If teachers are … Read More
student test scores are exactly that – the student’s. A different metric is needed or a variety of metrics. The formula to create test results varies year to year and is not relevant. Student acuity level should also matter just like doctors who treat more complex patients. Teachers should peer review themselves and actually evaluate each other honestly not just for their own benefit or out of fear. If teachers are evaluated by student test scores then parents should be held accountable too. Parents are a child’s first teacher right? From a parent with kids in school right now.
Jerry Heverly 9 years ago9 years ago
Here is something I wrote on this subject a few years ago:
http://patch.com/california/sanleandro/teaching-skill-is-subjective-not-easily-measurable
Gary Ravani 9 years ago9 years ago
Tom: As Linda Darling-Hammond has stated frequently" "We will not be able to fire our way to Finland." The expert (from UC Berkeley) who stated that the effectiveness of 1% to 3% of teachers was questionable stated that because he meant that it was such a tiny percentage that destabilizing the professional protections for the 97% to 99% was destructive to the system. The judge obviously and likely purposefully misconstrued the expert testimony. There are no studies … Read More
Tom:
As Linda Darling-Hammond has stated frequently” “We will not be able to fire our way to Finland.”
The expert (from UC Berkeley) who stated that the effectiveness of 1% to 3% of teachers was questionable stated that because he meant that it was such a tiny percentage that destabilizing the professional protections for the 97% to 99% was destructive to the system. The judge obviously and likely purposefully misconstrued the expert testimony.
There are no studies to document that firing teachers has resulted in greater student learning. There are only theories to that effect, the most popular developed by an economist (Saunders) whose specially was agricultural statistics.
Re the Darling-Hammond quote: There is a lot of work and a lot of resources that need to go to support schools, kids, and families to in the US to promote improved learning. The “short-cut” to it all, and the cheap alternative as always, of firing teachers leads us off a cliff. As the cheap alternatives, in dollars and integrity, always do.
Gary Ravani 9 years ago9 years ago
Just as the national movement against using state tests for high stakes decisions for students (let alone teachers) reaches a crescendo we get this. Just as people are beginning to pay attention to research done by the National Research Council, as well as many other legitimate researchers, that show an over reliance on test scores for students as well as teachers has actually negatively impacted learning nationally and narrowed curriculum in dangerous ways, we get this. Just … Read More
Just as the national movement against using state tests for high stakes decisions for students (let alone teachers) reaches a crescendo we get this.
Just as people are beginning to pay attention to research done by the National Research Council, as well as many other legitimate researchers, that show an over reliance on test scores for students as well as teachers has actually negatively impacted learning nationally and narrowed curriculum in dangerous ways, we get this.
Just as the information becomes available that the attempted continued abuse of teachers and their profession rights is negatively impacting teacher availability to districts, and a very possible teacher shortage that will force districts to hire less qualified teacher (interns and Teach for Awhile candidates), we get this.
Just as it has become quite obvious that Prop 30, a tax primarily impacting the very wealthy, is positively impacting CA’s schools and the ability to improve services to students we get this.
(Coincidently, “we get this” from the very wealthy who are impacted by Prop 30. And then there are the wealthy’s minions, the law firm mentioned above infamously known as the running dogs of corporations and defending said corporations from class actions suits related to corporate abuse of employees, the environment, and the public.)
Just as a deluge of research emerges about what qualifies as quality teacher evaluation, and the support systems needed to be put in place to do that, we get this.
Pay no attention to the educational realities behind that legalistic curtain, folks. Look over here at what the Wizards of Silicon Valley have to offer. Just don’t look too closely.
Replies
Concerned parent 9 years ago9 years ago
I disagree with your repeated mantra that . . TESTING BAD . . Assessments are good , especially for math, in some school districts the math assessments have been discontinued for more than two school years and no intentions of bringing them back in my opinion because common core math standards are being taught in wish washy ways of "ah ha," kind of coaxing of group child grading... . . . Testing is good, and quizzes, and assessments, and if teachers keep going the way … Read More
I disagree with your repeated mantra that
.
.
TESTING BAD
.
.
Assessments are good , especially for math, in some school districts the math assessments have been discontinued for more than two school years and no intentions of bringing them back in my opinion because common core math standards are being taught in wish washy ways of “ah ha,” kind of coaxing of group child grading…
.
.
.
Testing is good, and quizzes, and assessments, and if teachers keep going the way they are, the unions will be forced due to public lack of trust, to be changed, as we have witnessed in other states in the past few years…
.
.
So, I suggest you immediately triangulate your views if you want ant parent support.
Gary Ravani 9 years ago9 years ago
Concerned: You need to read more carefully. It's not my "mantra." It's the mantra of the highest scientific body in the nation, the National Research Council, as well as a great number of other legitimate educational researchers. The national obsession with state tests has slowed learning across the nation and dangerously narrowed the curriculum. Periodically you should attempt to leaven your opinions with some actual research on the subjects you approach with information from legitimate … Read More
Concerned:
You need to read more carefully. It’s not my “mantra.” It’s the mantra of the highest scientific body in the nation, the National Research Council, as well as a great number of other legitimate educational researchers. The national obsession with state tests has slowed learning across the nation and dangerously narrowed the curriculum. Periodically you should attempt to leaven your opinions with some actual research on the subjects you approach with information from legitimate sources.
My “mantra” is kids get assessed almost very day in a wide variety of ways and ways that actually count. Often described as authentic assessments and multiple measures, two different things by the way. The assessments (tests, quizzes, performances, projects, portfolios, etc.) are developed by the teachers in the classroom where the students are being educated and the assessments actually measure what is taught, not what some publisher a thousand miles away thinks should be taught.
Ann 9 years ago9 years ago
OK here’s the nrc’s pamphlet on assessment and it totally supports using tests as part of teacher evaluations. NOONE suggests it be the only means.
http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_082766.pdf
CONCLUSION
Tests are one objective and efficient way to measure what people know and can do,
and they can help make comparisons across large groups of people. However, test
scores are not perfect measures: they should be considered with other sources of
information when making important decisions about individuals.
Manuel 9 years ago9 years ago
Not surprisingly, my opinion is completely opposite to that of Zeev: this is an idiotic lawsuit because there is ample evidence that the VAM “yardstick” being promoted is useless.
But, hey, I am not a wealthy Ph.D. holder who backs standardized tests in all their surreal (and eugenic!) expectations like Welch, so what do I know?
Replies
Tom 9 years ago9 years ago
Dude, what is “VAM”? Something to do with vampires or what? Please enlighten us so that we can follow your argument. Thanks,
CarolineSF 9 years ago9 years ago
“Value Added Measurement” — the methodology for rating teachers based on fluctuations in their students’ test scores.
Tom 9 years ago9 years ago
Too bad school districts have to face expensive lawsuits to do the right thing = teacher accountability.
Replies
Gary Ravani 9 years ago9 years ago
Tom: Teachers' accountability? Really? CA is the nation's wealthiest state and its school funding has foundered in the lowest ranks of all 50 states for decades. Who is accountable for that? As a consequence of above, CA have the largest classes (=fewest teachers per student), fewest administrators, fewest librarians, fewest counselors, fewest nurses, and generally fewest number of adults per student in the nation. Who is accountable for that? The negative impacts of Prop 13 on all of CA's … Read More
Tom:
Teachers’ accountability? Really?
CA is the nation’s wealthiest state and its school funding has foundered in the lowest ranks of all 50 states for decades. Who is accountable for that?
As a consequence of above, CA have the largest classes (=fewest teachers per student), fewest administrators, fewest librarians, fewest counselors, fewest nurses, and generally fewest number of adults per student in the nation. Who is accountable for that?
The negative impacts of Prop 13 on all of CA’s public services was well recognized before it passed and has been confirmed through experience ever since. And yet it had great public support. Who is accountable for that?
CA has one of the highest child poverty rates, and poverty is closely correlated with low school achievement, in the US. Who is accountable for that?
The US has one of the highest child poverty rates of any of the major industrialized nations on Earth. Who is accountable for that?
CA’s school funding situation has measurably, and likely temporarily, improved and may actually get near the national average in dollars unadjusted for cost of living. Who can be held accountable for that? Actually, it’s the Democratic Governor, the teachers’ unions, and various community partners.
Accountability, in the larger sense and including all stakeholders would be great. Let’s have some finally.
Tom 9 years ago9 years ago
Really a tired argument Gary. California has the highest income tax, sales tax, gas taxes, payroll taxes, and there is plenty of money but the politicians choose to spend it on other things, e.g. high speed rail and overly generous unsustainable public employee pensions. Even in good times the politicians short-change our schools. Not holding teachers accountable via meaningful evaluations (and administrators) keeps the bad teachers in the classroom and hurts the … Read More
Really a tired argument Gary. California has the highest income tax, sales tax, gas taxes, payroll taxes, and there is plenty of money but the politicians choose to spend it on other things, e.g. high speed rail and overly generous unsustainable public employee pensions. Even in good times the politicians short-change our schools. Not holding teachers accountable via meaningful evaluations (and administrators) keeps the bad teachers in the classroom and hurts the kids. That needs to stop Gary, and it looks like it may take the courts to do it.
Deborah 9 years ago9 years ago
I agree the politicians short change the kids so why do teachers get evaluated based on the impact of those politicians? Do we get to apply VAM to politicians too? Teachers, kids and families stuck in the middle again. When will students really matter?
Gary Ravani 9 years ago9 years ago
Tom: Actually, CA is only a " high moderate" tax state, within the top 12 nationally depending on what you count. In cost-of-living weighted dollars CA is more near the middle of the 50 states. Since CA is second in the nation in cost-of-living that leaves a huge gap in what is available for spending on schools or other necessary projects. Just what would bond money for trains have to do with school funding? Most of the … Read More
Tom:
Actually, CA is only a ” high moderate” tax state, within the top 12 nationally depending on what you count. In cost-of-living weighted dollars CA is more near the middle of the 50 states. Since CA is second in the nation in cost-of-living that leaves a huge gap in what is available for spending on schools or other necessary projects.
Just what would bond money for trains have to do with school funding?
Most of the issues with pension funding in CA are the same as the issues that plague other states. Municipalities and counties took “pension holidays” and did not fund their systems. The state of CA was guilty of the same in some cases. But the fundamental reason was the recession driven by misfeasance and malfeasance in the private financial sector, for which none of the principals is being held accountable or being forced to compensate those they cheated. There was even illegal manipulation of the LIBOR, which cut earnings to pension as well as other collective funds.
There is a retirement crisis in the US. Too many people have no retirement plans or savings. This puts large segments of the elderly at risk. That problem will not be solved by undermining pension plans, it will only make it worse. The US (nor CA) doesn’t have too many people with secure pension and retirements, it has too few. Pensions for all (or at least most), as it was before the private sector unions were decimated, the wages of the middle-class were frozen, and income inequality grew to crisis proportions. The income inequality crisis has grown so bad even the GOP is making a effort to look like it is concerned. That’s saying something.
FloydThursby1941 9 years ago9 years ago
Cutting teachers’ pensions should be off the table. They need it for a middle class income. However, other pensions should be capped maybe at 90k. Police pensions are often fraudulent and many executive and UC Pensions are. Once you are retired 90k should be enough, maybe 100k. We do need to cut 200-300k pensions especially those based on false last year salaries.
navigio 9 years ago9 years ago
That was changed a few years ago.
Ann 9 years ago9 years ago
School funding has gone nowhere but up in real terms since Prop 13 and most of it 85% on average goes to salaries and benefits. The statistic that places us near the bottom includes the coefficient of high living costs, mostly housing.
Zeev Wurman 9 years ago9 years ago
Good. Very good. Excellent, in fact.