California, other states to set test cutoff scores

October 5, 2014

Students taking tests

During the next few weeks California educators will play a pivotal role in a crucial phase of work for the new Smarter Balanced assessments that millions of California students will take this spring for the first time: setting the cutoff  scores that will indicate whether a student is academically on track for the next grade level and ultimately whether they are ready for college and careers.

At the State Board of Education meeting last month, board member Sue Burr said setting what educators call “cut scores” to establish proficiency levels of students is “the next big piece of work” in the implementation of the Common Core standards adopted by the state in 2010.

Historically, states have chosen their own annual tests and set their own proficiency levels in assessing the performance of their students.  That will change this spring, at least for California and 16 other member states of the Smarter Balanced Assessment Consortium that are currently expected to administer the same new assessments, according to an Education Week analysis. The assessments are aligned with the Common Core, a set of academic standards adopted by nearly all states and representing potentially one of the most significant reforms in decades. More than 4 million students in those states – at least 3.2  million of them in California – in third through eighth grades and 11th grade will take the same assessments in math and English language arts.

The prospect of students in multiple states taking identical assessments, and officials in those states agreeing on common proficiency levels, is unprecedented in California and U.S. history, and many educators and researchers are welcoming the changes.

“I think it is critical that we be able to accurately compare the performance of California’s students with students in other states,” said Lawrence Picus, a professor at the USC Rossier School of Education. “In a country where I can travel nearly 3,000 miles and get an identical McDonald’s hamburger, it seems that I should also be able to expect that I can accurately compare the standardized test results from my child’s school with other schools and states across the country.”

The cut scores are important because they will provide guidance for anyone trying to interpret student performance on the new assessments –from parents and students to teachers, policy makers, education advocates and researchers.

“It’s critically important for us because it establishes expectations for student achievement,” said Diane Hernandez, director of the Assessment Development and Administration Division at the California Department of Education.

About two dozen teachers, test and curriculum administrators and college representatives from California will join nearly 500 educators from other states from Oct. 13-19 in Dallas to propose the cut scores for what educators call “achievement levels” for the assessments that are aligned with the Common Core state standards. They will also get input from thousands of volunteers from California who have signed up to be part of an online panel  that will take three hours on a day they have been assigned from Oct. 6-17 to review the tests and propose cut scores in a grade level and subject area of their choosing.

“In a country where I can travel nearly 3,000 miles and get an identical McDonald’s hamburger, it seems that I should also be able to expect that I can accurately compare the standardized test results from my child’s school with other schools and states across the country,” said Lawrence Picus, a professor at the USC Rossier School of Education.

The process will bring some states closer to actualizing a vision articulated exactly 25 years ago at an Education Summit that President George H.W. Bush hosted in Charlottesville, Virginia, with all 50 governors. The summit led to Bush’s America 2000 initiative, and its proposals for voluntary national tests. President Clinton also proposed voluntary national tests in fourth grade reading and eighth grade math. Both efforts foundered in the face of political opposition from a range of sources.

“There have been a whole bunch of shots at this over time,” said Marshall Smith, a former dean of the Stanford School of Education and a former high-ranking official at the U.S. Department of Education during the Clinton and Obama administrations.

With passage of the No Child Left Behind law, which President George W. Bush signed in 2002, the nation went in the opposite direction. Despite the detailed, top-down requirements imposed on states by the federal government, each state administered its own tests and set its own levels of proficiency. That made it impossible to compare how one state was doing in relation to another.

The adoption of the Common Core standards by 43 states and the District of Columbia, under the auspices of the Council of Chief State School Officers and the National Governors Association, has revived the idea of national standards in public education.

California has been a leader in the larger of the two consortia, the Smarter Balanced Assessment Consortium, which developed assessments that students will take in the spring. Nine other states and the District of Columbia, which belong to the Partnership for Assessment of Readiness for College and Careers, known as PARCC, will take a different set of assessments in setting their proficiency standards. Twenty-four states will administer other tests, including those drawn up by American College Tests (ACT) and the American Institute of Research, according to Education Week. 

The opportunity to participate in setting proficiency levels generated enormous interest in education circles in California, where about 800 teachers and other educators applied to be on the “in person” panel that will recommend where to set the proficiency levels on the Smarter Balanced assessments. Just over half of those who applied were deemed eligible to participate, and 27 were selected to join other educators in Dallas next week.  The names of those selected have not been released by the California Department of Education, although they will be published after the process is completed.

Each of the reviewers will take a practice test and be given an overview of the test-review process. Then participants will be assigned to smaller panels to review the assessments for each grade level and subject area, starting with the easiest questions and ending with the most difficult problems. Reviewers then will identify which questions they feel are most closely linked to a particular level of proficiency.

The Smarter Balanced consortium envisaged four levels of achievement, which describe whether students have demonstrated “deep command,” “sufficient command,” “partial command,” or “minimal command” of English language arts and math.

For more than a decade, California described student performance levels on the California Standards Tests, which students took until last year, as “far below,” “below basic,” “basic,” “proficient,” and “advanced.” The terminology that will be used to describe each performance level on the Smarter Balanced assessments – what educators call “achievement level descriptors” – has yet to be adopted.

Once the Smarter Balanced reviewers’ work is concluded, 60 panel members will break into two groups – math and English Language Arts – to ensure that cut scores at each grade level are appropriately aligned across each subject area.

Jacqueline King, director of higher education collaboration at the Smarter Balanced Assessment Consortium, said she was confident that the process will run smoothly. She said Smarter Balanced has conducted a “dress rehearsal” of the online panel and many of the experts attending the in-person panel meetings have been involved in achievement-level setting “many, many times before.” At the same time, an auditor will observe the entire process to prepare a report for the Smarter Balanced technical advisory committee.

Also providing input will be an online panel made up of thousands of educators, parents, community members and business people, two-thirds of whom are expected to be from California.  The closing date to sign up for the online panel was Sept. 26.

One online volunteer is Cindi Blair, a former elementary teacher in the San Bernardino City Unified School District who helps train the district’s test coordinators and examiners. Eight years ago, she participated in a panel that re-established cut scores for California’s standardized test, which measures proficiency levels of English learners. Two of Blair’s colleagues have also registered for the online panel.

Establishing common proficiency levels across states will be especially helpful when students transfer to her district from another state, Blair said. Comparing student achievement from other states has been an imperfect process, she said, especially at the high school level. Information on how students performed on the Smarter Balanced assessments will be helpful in getting students on the “same page academically,” she said.

The common proficiency levels have raised concerns about whether California would give up some of its autonomy by setting performance levels in tandem with 16 other states. “California had a great deal of autonomy on setting our achievement levels, and some of that power and autonomy is being ceded to the national consortium,” said Patricia Rucker, a member of the State Board of Education at its most recent meeting in Sacramento. Rucker was assured that the state would have an observer at the Dallas meetings – and that the state board would get a full report after the meetings concluded.

Others think these concerns are unwarranted. “That is a trade-off in an assessment consortium,” said Morgan Polikoff, a professor at the USC Rossier School of Education. “You have to believe that the benefits of setting common cut scores far outweigh the costs in terms of lost autonomy, which on this particular issue seems especially minor.”

The Smarter Balanced’s King said setting cut scores at a high level would be a departure from the current achievement levels of many states. However, she said the process will be more familiar for California since the state’s past assessment system established high cut scores, which in turn resulted in raising standards and expectations for students, and contributed to more students becoming prepared for college-level work.

Chief state school officers whose states will administer the Smarter Balanced tests will meet Nov. 6 to endorse the proficiency level recommendations coming from the Dallas meetings. The recommendations must be approved by the state board of education in each member state. In California, the proficiency levels will be considered during the State Board of Education’s Nov. 13-14 meeting in Sacramento.

A question raised by the entire process is whether setting the same proficiency levels across states will make a difference in improving student outcomes, so that students leave school ready for college and careers – the ultimate goal of the Common Core standards.

Former U.S. Under Secretary of Education Marshall Smith said it will be crucial for schools “to have the resources and support to implement the Common Core” along with the Next Generation Science Standards, a similar effort to establish national standards for science education in K-12 schools. “If we can get decent responses from teacher training institutions, if we understand it will take six, seven or eight years” to fully implement the new standards, “and keep working on it, it will be terrifically important,” Smith said.

Karla Scoon Reid contributed reporting for this story. 

To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.

Share Article