Proceed with caution when comparing California test scores with other states

CAASPP sbac smarter balanced
Students in California preparing for the Smarter Balanced tests.
Credit: Berkeley Unified School District

Update: This story was updated on Sept. 11, 2015 to include Maine’s scores, and on Sept. 27 to include Oregon’s scores. 

Comparing California scores on tests aligned with the Common Core standards to those in other states isn’t a straightforward process.

California students’ results are among the lowest when compared to the other eight states that have released Smarter Balanced assessment scores so far. But drawing conclusions may be difficult because California’s student population is much larger and its schools enroll more English learners and low-income students. See charts of scores in eight Smarter Balanced states.

“It’s not just a straight across comparison,” Keric Ashley, California’s deputy superintendent of public instruction, warned during a Wednesday conference call with reporters. “We need to factor in being such a large state as we are – a large percentage of English learners, a large percentage of students in poverty. There are a lot of factors that go into place before making that comparison.”

The chart shows a comparison of 3rd grade English language arts results on the Smarter Balanced test in nine states. Click on the graphic to see more results comparing California to other states.

EdSource

The chart shows a comparison of 3rd grade English language arts results on the Smarter Balanced test. Click on the graphic to see more results.

One of the main arguments in favor of the Common Core standards, and the standardized assessments aligned with them, was that for the first time it would be possible to compare annual student performance across dozens of states.

States had the opportunity to administer tests developed either by the PARCC consortium or the Smarter Balanced Assessment Consortium, which California chose. Some states ultimately chose other tests.

A total of 17 states administered the Smarter Balanced assessments, and eight have released preliminary or final Smarter Balanced scores: Connecticut, Delaware, Idaho, Missouri, Oregon, Vermont, Washington and West Virginia. The remaining states will release their scores later.

UPDATE: On Sept. 11, Maine became the 10th state to release its Smarter Balanced Assessment scores with 48 percent scoring proficient or above in English language arts and 36 percent scoring proficient or above in math. Maine plans to drop Smarter Balanced tests this school year.  On Sept. 17, Oregon released its results, indicating that 54 percent of students scoring at a Level Three or Four in English language arts, and 40 percent at Levels Three or Four in math.

The Smarter Balanced scoring system is the same in every state, ranging from level 1, the lowest, to level 4, the highest – with the same cut scores set for each level – but each state has its own terminology to describe the different levels.

Generally, a level 3 or 4 means a student has met or exceeded the standards. In most states, level 3 is labeled “proficient,” while Connecticut calls it “meeting the achievement level.” In Oregon, it’s simply listed as a 3 or higher on the state’s press release regarding preliminary scores. In California, policy makers have avoided using the word “proficient” in labeling its scores, instead calling level 3 “standard met” and level 4 “standard exceeded.” See descriptions of the score levels.

Not all states have released composite scores for students from all grades who took the tests – grades 3-8, and grade 11 – making it harder to easily compare performance.

But looking at the results that are available from those eight states, most reported a higher percentage of students who scored at a level 3 or 4 in most grades and subjects compared to California.

Compared to the eight states, the percentage of California students who scored at proficient or above was the lowest or tied for lowest in 3rd- through 5th-grade math and 3rd- through 6th-grade English. California ranked fourth from the bottom for 11th-grade math and English. (Missouri 11th-graders didn’t take the Smarter Balanced test.)

But the lower scores in California may be due to the diversity of its student population. Andy Latham, director of WestEd’s Center on Standards and Assessment Implementation, said states with large numbers of English learners and low-income students, like California, will tend to have lower overall scores.

“It could be an explanation why proficiency levels are not as high as we’d hope, but that shouldn’t be an excuse,” Latham said.

The combined number of students in the eight states was about 3.9 million in 2012-13, according to the most recent data available from the National Center for Education Statistics. California’s student population was 6.3 million that year, according to the center.

English learner students made up well under 10 percent of those state’s enrollment in 2012-13. In California, it was 23 percent that year.

Los Angeles County has more English learners – about 350,000 in recent years – than those eight states combined: about 226,000 in 2012-13.

English learners tend to score lower on tests because they are still mastering the language. Many of them are “reclassified” out of the English learner category after they score high enough on standardized tests and those reclassified students fared better on Smarter Balanced assessments.

Also, some English learners may get extra help, such as test directions or a translation glossary in Spanish. English learners who have been in the U.S. for less than a year are exempt from the English language arts test in California, but they still must take the math portion. More than 600,000 English learners, out of 3.2 million students, took the Smarter Balanced tests in California. Of those, 11 percent met or exceeded the standard on both English and math tests.

“It could be an explanation why proficiency levels are not as high as we’d hope, but that shouldn’t be an excuse,” said Andy Latham, WestEd’s director of Center on Standards and Assessment Implementation.

Latham said it would be better to compare California’s English learners’ scores to those students in other states. Only four of the states released some of those English learner results so far.

Also, Ashley warned that each state may give the tests in slightly different ways, such as exempting certain types of questions, making it tougher to compare.

Vermont’s scores for English learners varied widely, depending on the grade level and subject. For example, in English, the percentage of English learners scoring proficient or above ranged from 36 percent in 3rd grade to less than 2 percent in 11th grade.

But Vermont’s number of test takers is tiny compared to California’s: A total of 552 English learners took the Smarter Balanced math test in the entire state. That’s less than the number of English learners in a single Santa Ana high school.

Connecticut released a combined score for “high needs” students, which it defines as the combined number of those eligible for free or reduced price meals and English learners. Of the English learners, the state tested those who have been in the country for at least a year. About 31 percent of “high needs” students scored proficient or above in English language arts and 16 percent in math.

In Missouri, where about 16,000 English learners took tests, about 37 percent were proficient or above in English language arts and 28 percent were proficient or above in math.

Next to California, Washington had the second-highest number of English learners tested – about 47,000. The percent proficient or above on English language arts ranged from 10 to 19 percent, depending on the grade. Because about half of the 11th-grade class opted out of the tests, there were too few students in that grade scoring proficient on the English language arts test for the state to release results.

EdSource in your inbox!

Stay ahead of the latest developments on education in California and nationally from early childhood to college and beyond. Sign up for EdSource’s no-cost daily email.

Subscribe