District officials want to avoid overreacting to new test results

September 30, 2015

Before the pandemic, students work in a Santa Ana Unified classroom in a way that would be unimaginable today.

As educators across the state examine the results of the Smarter Balanced assessments that millions of students took last spring, officials in several school districts that EdSource is tracking say they want to avoid overreacting to the scores and that they want to take more time to review the results before significantly adjusting what they are already doing.

“We’re not going to stop all activity and shut everything down and recalibrate right now, given that we’re coming into our third week of school,” said Fresno Unified Superintendent Michael Hanson after the state released the scores Sept. 9. “I feel good about some bright spots. But, candidly, I don’t know what they mean and why. It will take a few years to iron out.”

District leaders said the Smarter Balanced scores set an important baseline to assess future academic growth. But several said they are also focusing on improving performance in other areas and are still waiting for more detailed results that could give them a better idea of specific content areas in which students excelled or fell short. In general, EdSource interviews suggest the test results have yet to inform instruction in a systematic way.

The Smarter Balanced test scores are among more than two dozen items the state will use to assess school success, including school climate, attendance and graduation rates, and reducing suspensions and expulsions.

“It’s one of 30 different measures we’re using, from attendance to suspension rates,” said Elk Grove Unified superintendent Chris Hoffman. “There’s a whole range of measures that we think are important in determining kids being successful.”

The responses reflect the state’s new school local control and accountability reforms promoted by Gov. Brown and enacted by the state Legislature, which downplay the importance of test results alone and place a priority on a far more comprehensive set of measures of school and student performance.

Fresno Unified and Elk Grove Unified are among the half-dozen districts EdSource is following as they implement the Common Core standards. The others are Garden Grove UnifiedSanta Ana UnifiedVisalia Unified and San Jose Unified. Also in the sample are the Aspire Public Schools, with 35 charter schools around the state.

Rather than scores being used principally to rank and rebuke schools, teachers and districts – which was a feature of the No Child Left Behind era of reforms – officials are hoping that the new scores will actually help improve student performance.

Jason Willis, San Jose Unified’s assistant superintendent for community engagement and accountability, said that his district wants to “use the recent results as an additional tool to help our students. The opportunity Smarter Balanced creates is just one more data point in a set of data for us to be able to learn and understand as we work toward the outcomes that we desire, for all of our students to be successful.”

A frustration expressed by officials in some districts is that they received scores later than expected, and what they received was much less specific than they had hoped for. That has made it more difficult to adapt their teaching strategies or other interventions for individual students based on their test performance.

“One of the challenges has been wanting to get more detailed information on the results, which we don’t have now,” said Garden Grove Unified Superintendent Gabriela Mafi. “I guess it’s useful as part of the state’s accountability system, but teachers are used to much more discrete content cluster level data, where they can look particularly at vocabulary, word analysis, and they can also see the scope of the things that were tested.”

Whereas districts received particulars about what was tested under the previous California Standards Tests, she said they have only been given very general information about what was assessed on the new California Assessment of Student Performance and Progress, known as CAASPP.

“So, within ‘reading,’ we have no breakdown of which parts of reading are better or worse,” Mafi said, adding that CAASPP results are “not specific enough to guide instruction in any way.”

Another challenge, Mafi said, was that students could choose multiple correct answers on some Smarter Balanced test items. “We don’t know how well kids did on those types of questions because we don’t have that fine-grain data,” Mafi said.

A further obstacle is that teachers do not have direct access to their student’s results, as the state is still developing an Online Reporting System that will allow that.

Before adapting what they are currently doing, some officials also want to look at how or why students in similar districts did better on the tests, and possibly consider adopting practices implemented elsewhere.

“There is a lot of sharing that’s happening between school districts of similar size and student demographics,” Willis said. “We find value in sharing effective practices and learning.”

That will involve having frank discussions about where districts are succeeding, and where they are not, he said. “When we get into conversations with other districts, we really want to create an opportunity to be vulnerable,” Willis said. “As professionals, we want to create an environment where people can really be honest and open about what’s happening. That’s the first great step to getting better as an institution.”

One reason that district officials are finding it challenging to interpret the results is that for the first time this year, students who answered a question correctly were given more difficult computer-generated questions to answer. If they answered incorrectly they were given easier ones.

The purpose of these “computer adaptive” tests was to drill down to find out more precisely what individual students knew or didn’t know.

“Part of what we really have to figure out is how detailed the information is that we’ll receive compared to what we used to get with the California Standards Test (administered to students until 2013) to dig down to specific items and details within the tests,” Elk Grove’s Hoffman said.

He said the scores provide useful information – especially to see how his district compares with others – but, “it’s not something we’re using on a day-to-day basis in intervening with kids.”

Among the six districts being tracked by EdSource, student performance on the tests varied considerably from district to district (see tables below).

San Jose Unified had the highest percentage of students – some 51 percent of test takers – who met or exceeded standards on the English language arts portion of the test. About 39 percent met or exceeded standards on math.

By contrast, Santa Ana had the lowest percentage of students – 25 percent – who met or exceeded the standards on the English language arts portion of the test. About 21 percent met or exceeded the standards on math.

But it is worth noting that San Jose also had one of the lowest percentages of English learners (17 percent) and the lowest percentage of students who qualified for free and reduced-price meals (45 percent) among the 3rd through 8th graders and 11th graders who took the tests. Santa Ana, by contrast, had the highest percentage of English learners (4o percent) and the highest percentage of students who qualified for free or reduced-price meal programs (93 percent).

In all districts, there were substantial gaps between the scores of Asian-American and white students on the one hand and black and Latino students on the other. But officials said the achievement gaps underscored by the results don’t come as a surprise to them, and that they all have had plans in place to address them.

One reason for the gap is that many students do not have access to computers or the Internet in their homes, putting them at a disadvantage when it came time for testing, said Santa Ana’s Lucinda Pueblos, an assistant superintendent for school performance and culture.

Santa Ana plans to continue adding technology to reach a wider range of students and is using Local Control Funding Formula dollars to add more student intervention programs, teacher training and other services to help more students succeed, Pueblos said.

Some districts have responded to the test scores with modest measures to target resources at schools that need the most help. San Jose, for example, is trying to concentrate instructional coaches – teachers who have expertise in an academic discipline or working with special populations like English learners – in schools where students received the lowest scores.

“We have through the Local Control Funding Formula more resources on those campuses with higher percentages of low-income students, English learners and foster youth,” Willis said. The goal is to ensure “we’re providing quality opportunities for every one of our students to be college and career ready.”

Similarly, Visalia Unified will evaluate its Local Control and Accountability Plan goals and reallocate resources next year based partially on the test results, said Superintendent Craig Wheaton.

“This gives us information that we can mine and find the nuggets and change our practices or reinforce our practices,” he said.  Test results show that in particular, English learners need the most help, and math is a subject where more students are lagging. “I think it will help inform and direct our goals as we move forward,” he said.

Aspire’s charter school network is looking at campuses where students performed at the highest proficiency levels on the Smarter Balanced tests in order to possibly replicate successful programs at its other schools.

“We plan to do regression analysis to identify which schools outperformed what would be predicted – based on numbers of students on free and reduced lunches and English learners – to look at those schools beating the odds,” said Elise Darwish, Aspire’s chief academic officer. “We’re doing a lot of autopsy work.”

Even as they look closely at this year’s results, Fresno’s Hanson said his district will avoid “teaching to the test” when students prepare to take the Smarter Balanced assessments again. “You’re not going to see bumper stickers in Fresno that say, ‘Give it your best for Smarter Balanced next spring,’” he said. “It’s important, but it’s not a driving factor.”

Instead, Hanson said his district is spending a lot of its resources on efforts such as improving school climate and field trips to a science camp, museums, national parks and other world-class destinations such as the Monterey Bay Aquarium to provide students with the same kinds of life experiences those in more affluent areas receive. The district has also purchased musical instruments, is offering a wider variety of clubs and programs such as “History Day,” has adopted new math materials, is providing curriculum training to teachers and administrators, and has lengthened the school day at several campuses.

Some districts are looking to the state to provide further guidance on how to use the test results to inform instruction. To that end, they plan to send teachers and other educators to workshops or “institutes” that the state is sponsoring in October.

“We’re going to be participating in the institutes,” Mafi said. “We’ve been proceeding with caution in interpreting the scores, only because comparisons to the previous system are so challenging. We just don’t know enough yet.”

Staff writers Fermin Leal and Sarah Tully contributed to this report.

To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.

Exit mobile version