Opinion > Commentary

D.C. cheating issue calls test-driven incentives into question


Charles Taylor Kerchner

Charles Taylor Kerchner

The smoke surrounding allegations of test score cheating in the Washington, D.C., public schools burst into flame last week. In a 4,300-word blog post, titled Michelle Rhee’s Reign of Error, the veteran educational journalist John Merrow linked the former schools chancellor with documents that suggest that she knew about widespread cheating on standardized tests and looked the other way.

Merrow drew parallels between Washington and Atlanta, where former superintendent Beverly Hall and 34 others have been indicted. But the underlying question is whether school reform can successfully be driven by rewards and punishments tied to standardized tests.

First, the Watergate question: What did Rhee know and when did she know it? As U.S.A. Today reported, District of Columbia Public Schools officials have long maintained that a 2011 test-cheating scandal that generated two government probes was limited to one elementary school. Merrow, however, reported that a long-buried memo from an outside data consultant warned as far back as January 2009 that educator cheating on 2008 standardized tests could have been widespread, with 191 teachers in 70 schools “implicated in possible testing infractions.”

Consultant Fay G. Sanford noted substantial numbers of erasures on test forms where corrections were made from wrong answers to correct ones. (Erasers leave smudge marks that the machines doing the grading recognize along with the answers.) “It is common knowledge in the high-stakes testing community that one of the easiest ways for teachers to artificially inflate student test scores is to erase student wrong responses to multiple choice questions and recode them as correct,” Sanford wrote.

When asked, Rhee said that she didn’t recall getting a memo from Sanford, an assertion she repeated this week to the Los Angeles Times editorial board.

A day after the Merrow report was issued, another investigation was released citing “critical” violations of test security in 18 classrooms located in seven district schools and four charters. The test was given in more than 2,600 classrooms, and current D.C. schools chancellor Kaya Henderson said in a statement, “We are pleased that this is yet another investigation that confirms that there is no widespread cheating at DCPS.”

But Henderson is not quelling demands for a district-wide investigation. Randi Weingarten, president of the American Federation of Teachers, has called for an independent investigation, as has Washington Post writer Valerie Strauss, who said, “The memo does not offer conclusive evidence that cheating occurred, but it literally begs for a thorough probe to be conducted—this time by investigators with subpoena powers.”

Meanwhile, an in-depth evaluation of D.C. schools by a National Research Council panel continues. The study is chaired by two Californians: Carl Cohn, a professor at Claremont Graduate University and member of the State Board of Education, and Lorraine McDonnell, a professor of political science at U.C. Santa Barbara. Their work is not confined to test tampering, but considers the overall effect of governance and management changes.

Test tampering only the symptom

Education policy organizations that have generally supported Rhee were quick to minimize the test tampering as “bad judgment.” Michelle Gininger at the Thomas B. Fordham Institute cautioned that: “Critics are bound to say that all reforms that Rhee stands for — teacher evaluations, tenure reform, and school choice — should be dismissed. But let’s not abandon education reform and accountability at the expense of our students, which getting rid of testing surely would.”

But, along with her fear-based management style, reliance on test scores is the Achilles’ heel of Rhee’s reform policies. Accurate test scores form the basis for her performance-based pay system and means to determine which teachers and administrators should be kept or fired.

Unfortunately, the system Rhee advocates inherently motivates people to cheat. As Cornell Law School professor Lynn Stout wrote in the Los Angeles Times, “It’s almost impossible to design objective performance metrics that can’t be met through illegal or undesirable behavior….  And when you create a system that inadvertently incentivizes illegal or undesirable behavior, you get more of it.”

This is the lesson of organizational history, not an isolated “bad judgment” aberration. It’s about more than school test scores in the District of Columbia, Atlanta, Texas or even Rhee’s possibly outsized claims of how well her students did during the three years she taught school in Baltimore. The policies Rhee endorses create bad incentives. Bad incentives lead to disastrous results. They certainly played a part in the largest business collapses in recent history: Enron, WorldCom, Lehman Brothers and the collapse of the subprime mortgage market.

The antidote is not to abandon incentives tied to test scores and other measures of achievement, but to use them in ways that actually motivate teachers, and more importantly students, who are the real workers in the education system.

What motivates teachers most? Student success: If an organizational system of curriculum, pedagogy, professional training and school organization helps students experience success, then teachers are highly motivated. Teachers are motivated by being part of a winning team, a school that does well at its own mission, which most often is not test score maximization. Teachers are motivated by being part of an occupation that is honored and trusted. These are the lessons from a century of study.

How should we then use achievement data? First, build feedback into the lesson and project level of schooling rather than waiting for end-of-year tests. Second, use tests that actually measure student performance on substantive tasks. In other words, build data feedback into the everyday life of teaching and learning.

What policies use data effectively?

  • Policies that build time into the school day for teachers to work together to interpret student results and self-correct;
  • Policies that invest in smart software that adapts to student performance and build computer routines that augment teacher effort;
  • Policies that build capacity into the education system.

This real road to student success is substantively at odds with Rhee’s policies. The difference is no small matter. Rhee’s StudentsFirst organization is attempting to create a political counterforce to the existing institution of public education and most current ideas about improving student achievement. In the words of Rhee’s recent book, its ideas are, indeed, Radical. Although it refers to itself as a nonprofit, giving StudentsFirst the public gloss of a thoughtful policy institute, it operates as a political organization. The Internal Revenue Service classifies it as a 501(c)4 organization, which means that contributions to it are not deductible on individual or business tax returns.

StudentsFirst is gathering supporters. Mayor Antonio Villaraigosa and former state senator Gloria Romero have signed on, as has Nevada Governor Brian Sandoval and Rhee’s mentor, former New York schools chancellor Joel Klein. Bill Cosby and Connie Chung are among its celebrity board members. Subsidiary organizations have been founded in several states.

StudentsFirst has started grading states. California received an F for its educational policies. (This same ranking grades Louisiana a top-of-the-pack A, and gives Massachusetts, which leads the world on some international measures, a D+.)

StudentsFirst also poured $250,000 into the recent Los Angeles Unified School District races, and supported candidates in Burbank and suburban Sacramento.

More than the question of test score cheating, it’s the logic and practice of Rhee’s policies and politics that deserve examination. To use the descriptors Rhee favors, they just may be “crappy” and “suck.”

• • •

Charles Taylor Kerchner is Research Professor in the School of Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy and teachers unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District.

Filed under: Commentary, State Education Policy, Testing and Accountability

Tags: , , , ,

Comments

EdSource encourages a robust debate on education issues and welcomes comments from our readers. The level of thoughtfulness of our community of readers is rare among online news sites. To preserve a civil dialogue, writers should avoid personal, gratuitous attacks and invective. Comments should be relevant to the subject of the article responded to. EdSource retains the right not to publish inappropriate and non-germaine comments. EdSource encourages commenters to use their real names. Commenters who do decide to use a pseudonym should use it consistently.

Leave a Comment

Your email address will not be published. Required fields are marked *

 characters available

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

15 Responses to “D.C. cheating issue calls test-driven incentives into question”

  1. navigio said

    on April 22, 2013 at 2:16 am

    You see, the problem with politics is that its ‘ok’ to get the right answer even if you do it the wrong way. The true goal is not student improvement, rather political reforms. The fact that Massachusetts gets a D+and Michelle Gininger still defends the ends over the means is proof positive of that.

  2. navigio said

    on April 22, 2013 at 7:43 am

    Last night former Atlanta superintendent of schools Beverly Hall, along with 35 teachers, principals and others, were indicted for racketeering. The core “criminal” activity alleged is that teachers, principals and test administrators, either under Hall’s explicit direction or thanks to a “climate” that endorsed such behavior altered the results of hundreds, or thouands of standardized tests given to Atlanta’s public school children.

    and..

    Hall inculcated an atmosphere that encouraged using any means necessary to achieve test-score targets, the indictment said, and then “publicly misrepresented the academic performance of schools throughout APS.” Pressuring subordinates to produce targeted scores, the indictment said, “created an environment where achieving the desired end result was more important than the students’ education.”

    Sounds like NCLB. Or CA. Or any system that is designed to measure achievement but used to measure teachers or administrators or schools.

    Now Hall, 66, faces as much as 45 years in prison.

    Gulp. So are we paying attention now?

    • twinkie1cat replied

      on May 10, 2013 at 11:11 am

      The cheating problem in Atlanta was going on well before Beverly Hall, who was brought in to reduce some of the patronage and cronyism that permeated Atlanta Public Schools. APS had such a problem with that that teachers were promoted and retained based on where they went to college and church and their Greek affiliation as well as their race and who they were related to, that there were a number of very incompetent and crooked administrators, especially female principals and male principals who seduced teachers of choice. I was aware of the cheating in 1990 when the school where I was working inflated its standardized test scores so much that they lost their eligibility for Title 1 money. I believe the test being used at the time was the Iowa. One of the paraprofessionals talked about seeing teachers giving answers to students. The students were retested so the school could get its grant. The next year “spies from downtown” as the principal called the monitors, were sent out to ensure the testing was done properly. There was always also a big hoopla in APS when the special education students scored better than the regular education students. Thing was, special education did not have to cheat. They used legal means to help their students to pass the tests, such as the proper use of 504 accommodations and teaching the mess out of the concepts that would be taught on the test.

  3. CarolineSF said

    on April 22, 2013 at 9:22 am

    And why Hall and not Rhee…?

    • navigio replied

      on April 22, 2013 at 9:55 am

      Money and politics most likely. Google those names and Bruce Dixon for one take.

      • CarolineSF replied

        on April 22, 2013 at 1:27 pm

        John Merrow did a piece asking the question (paraphrasing) “who made Rhee”? Among others, he blamed himself (for many segments, presumably non-hard-hitting, about her). He also blamed teachers’ unions. This made a lot of people furious, but it did insulate him from the usual charges of “teachers’ union lackey,” now that he’s the primary voice calling Rhee to account.

        By comparison, Hall was a nobody, of course. So she takes the fall while, so far, Rhee is still the reform diva.

        In my opinion (which I’m well aware others will dispute), being middle-aged (born 1948) and black (Jamaican-American) make Hall an easier target too.

  4. Fred Jones said

    on April 22, 2013 at 11:34 am

    The problem with Mr. Kerchner’s recommendations is the high cost of such substantive tests. Policymakers have attempted to hold districts accountable “on the cheap” (standardized tests of really only two academic disciplines — and a narrow bandwidth of those). But “accountability” is way too popular a political notion to simply throw-out any form of high-stakes testing and other accountability indices. The trick is making the tests more meaningful without expending a lot of funds (and classroom instructional time) to do it.

    Right now, however, we have a system that this quote aptly describes (from a retired CSU professor):

    “What was educationally significant and hard to measure has been replaced by what is educationally insignificant and easy to measure. So now we measure how well we taught what isn’t worth learning.”

  5. Manuel said

    on April 22, 2013 at 1:26 pm

    In my opinion, this is the most important part of Merrow’s report:

    “Although Rhee removed about 100 central office personnel in her first year, the central office today is considerably larger, with more administrators per teachers than any district surrounding DC. In fact, the surrounding districts seem to have reduced their central office staff, while DC’s grew.[32] The greatest growth in DCPS has been in the number of employees making $100,000 or more per year, from 35 to 99.[33]Per pupil expenditures have risen sharply, from $13,830 per student to $17,574, an increase of 27%, compared to 10% inflation in the Washington-Baltimore region.”

    Given that LAUSD’S budget has remained constant in dollars, has lost enrollment and, therefore, reduced the number of teacher positions, and, more importantly, no one has gotten a raise in years, I’d wager dollars to donuts that the same thing is happening under Deasy’s reign at LAUSD. With the exception of increasing per pupil expenditures, of course. Follow the money!

  6. Arnie said

    on April 22, 2013 at 8:43 pm

    Test-driven incentives lead to otherwise honest teachers cheating for personal gain only if there isn’t a way to easily detect such activity. Since the ways of catching such cheats isn’t fool proof, test-based incentives are worthless, not flawed. Worthless. I feel like everyone is shocked that teachers would stoop this low. Teachers respond to incentives just like everyone else. Reduce the stakes and you’ll reduce this behavior.

    Rule enforcement exists for a reason. If enough is on the line, there is always going to be someone who can’t live by them. I wouldn’t be putting any teachers in jail over this. The system designers are who should be called to account for not understanding very basic human behavior.

  7. Paul Muench said

    on April 23, 2013 at 6:03 am

    How are students harmed by teachers cheating on standardized tests? Would they be denied important services because they scored too well on a test? Do students students work less hard because success comes easier? Can we take our general distaste for test cheating and tell a story of concrete harm to studemts?

    • navigio replied

      on April 23, 2013 at 7:38 am

      There are schools in CA that use CST results as one indicator of intervention need. Similar for CELDT. Other states may use these differently, but in theory, yes, if they are used that way, false positives would harm students by denying them intervention. Probably more likely is the impact it has on a parent’s perception of student achievement level. Ongoing assessments can be notoriously variable, so some place more importance on the standardized test. One of the biggest causes of anger in the school system seems to be when a parent learns their student was not really at a level they thought they were (can be based on standardized tests or other things such as teacher feedback). Most often noticed when switching schools.
      Also, for anyone who believes that accountability has validity, a cheating teacher could be less likely to bother actually teaching. And less likely removed for performance reasons (obviously whether those are valid uses of standardized tests is not agreed upon).
      Ironically, by some arguments, cheating can also help students. There are teachers who claim testing hurts kids, both the process as well as how it impacts classroom time. If teachers cheat so they can instead do what teachers should be doing, then that would, in theory, help students.
      Obviously, there are also all sorts of other legal and moral problems with cheating that dont have to do with the students directly (performance pay, false trust in ability and wrong policy, fraud, etc, etc).
      was your question a trap? ;-)

      • Paul Muench replied

        on April 23, 2013 at 8:58 am

        No.

  8. CarolineSF said

    on April 23, 2013 at 7:53 am

    Somewhere in all the coverage of *alleged* cheating in D.C. schools (there’s been quite a bit of coverage, though it doesn’t seem to stick), I’ve seen quotes from parents who raised questions when their kids came up with high test scores in areas where the parents were aware they had problems. So it does seem that obscuring academic needs would be an effect of cheating.

    On the other hand, if we accept that closing schools is highly disruptive to students and communities — especially the most vulnerable, high-need students and communities — cheating in the hope that the school won’t be punished could be seen as beneficial to students. The questions this all raises about attaching high stakes to testing may not need to be pointed out.

    • twinkie1cat replied

      on May 10, 2013 at 11:20 am

      Because she is not a teacher and no one except teachers, and possibly other “calling” based professions such as the ministry,understand that we don’t teach to make money. We teach because we are teachers and would do so even if we were not making money at it. Providing education is an end, not a means to an end. A lot of other people in high places, including, apparently President Obama and Arne Duncan, having bought into Rhee, also don’t comprehend.

Template last modified: