With California officials warning that scores on Smarter Balanced tests will likely be lower than those on previous standardized assessments, many teachers want to know how they can better prepare students.
To help demystify the Common Core-aligned tests, the Educational Testing Service, or ETS, is providing several scoring training workshops for teachers this month.
The first Summer Scoring Training for Educators in Sacramento attracted more than 100 teachers and district administrators interested in learning how portions of the tests that are not multiple-choice are hand-scored.
“A lot of people want this information to take back to their schools,” said Jeff Grove, ETS mathematics assessment director. “A lot of people don’t realize how much is involved.”
The training focused on “performance tasks” students were asked to complete in mathematics and English language arts. These tasks involve classroom instruction and generally require longer responses than questions that are scored by computer.
For example, a math task could involve a scenario that requires students to make change, figure out the volume of an object, interpret a graph or table, or make calculations based on a word problem. An English language arts task could require students to read three passages about a certain topic, then write a short essay or story based partially on the sources provided.
“STAR tests didn’t have that,” Grove said, referring to the state’s previous standardized tests. “This is very new for teachers and students, so it’s important.”
The training included PowerPoint presentations, scoring rubrics and samples of real student answers annotated with explanations about why they received specific scores.
Although districts are gearing up for the release of Smarter Balanced scores from the first tests given statewide last spring, ETS representatives said they hoped the training would prompt more districts to use interim assessments that can be administered during the year. These optional, shorter tests are scored by district teachers and can help educators and their students to practice test-taking strategies and gain a better understanding of what differentiates high and low scores.
Some teachers and administrators who attended the training said in later phone interviews that they planned to share what they learned with others in their districts to give educators a better sense of elements that are heavily weighted.
“I really wanted to find out across California samples of some of the answers students gave so I’ll know how to improve my teaching of writing,” said Alanna Butterworth, a 3rd-grade teacher at Camellia Basic Elementary School in the Sacramento City district. “What I took away is that the kids really need to know keyboarding and writing.”
The Sacramento district administered interim tests during the past school year and trained teachers including Butterworth to score them according to Smarter Balanced rubrics, she said. The ETS training ￼workshop, Butterworth said, confirmed that she was scoring correctly by adhering strictly to the rubrics.
According to the rubrics, “effective” elaboration and development merits a 4, “adequate” elaboration and development rates a 3, “uneven, cursory elaboration and development” should get a 2, while “minimal elaboration and few or no details” should receive a 1. Those who don’t understand the content, don’t answer or answer in gibberish receive a 0, Grove said.
Butterworth said some high school teachers in her group felt the rubrics were a bit rigid.
“There was a great deal of disagreement when we tried to come to a consensus about whether this (sample answer) was a 3 or a 4” she said. “High school teachers were questioning, ‘What is the difference between adequate and effective?’”
However, Michelle Campbell, who teaches 8th grade English at Katherine L. Albiani Middle School in Elk Grove, said her group – which included both elementary and secondary teachers – thought the rubrics were very clear.
“Once we got through our first couple (of samples) and talked about them, we were pretty well calibrated,” she said. “We went back to the annotations to see what made a 4 a 4 or a 3 a 3. We thought it was consistent and we really liked that.”
The Elk Grove Unified School District sent seven teachers to the training, along with two principals and a vice principal. The nearby San Juan Unified School District sent research specialist Lisa Campbell, a former middle school math teacher, since teachers were unavailable due to other professional development.
San Juan used interim assessments last year with selected teachers who were willing to try them, Lisa Campbell said. This year, she said the district plans to train one teacher from each school to score the tests, with the expectation that they will train colleagues at their sites according to the “training of trainers” model.
“We’re scheduling trainings where teachers come and we’re going to give them one of the performance tasks and they will all come together and grade it,” she said. “We want teachers to feel free to use (the interim assessments) so they can use the data.”
ETS did not allow EdSource or teachers who attended the training to reveal the performance tasks or questions for security reasons.
The math performance tasks included six questions based on a common scenario or word problem. In some cases, students were asked to use answers from one question to figure out answers to subsequent questions.
If a student makes a mathematical error in answer to one question, then uses that incorrect answer to respond to another question that is reasoned correctly, the student will get full credit for the subsequent question, Grove said.
English performance tasks included “full write” items worth 10 points and “research short text” items worth 2 points. Full write answers involved narrative, opinion/argumentative, and informative/explanatory writing. Research short text items required short responses to questions that asked students to justify their answers and cite their sources.
After seeing how tests were scored, Lisa Campbell said teachers might tweak their instruction by insisting on source citations in English language arts lessons and on clear explanations of calculations in math.
Last year, Butterworth said her 8-year-old students really needed to practice finding the apostrophe key and holding down the shift key to make capital letters and question marks. For math problems, they had trouble finding symbol keys and clicking and dragging items from one place to another on their computer screens, she said.
Butterworth said the Smarter Balanced description of “elaboration” would help her to explain it to students. It includes “relevant details, dialogue, and/or description,” also known as the “three Ds in narrative writing.”
Michelle Campbell said the training would inform her practice.
“I think I’m a better teacher when I know how students are assessed,” she said. “For me, it validates the things I’m already doing. I’m on the right track.”
It was helpful, she said, to understand the individual elements that made up the 10-point scores. For example, she appreciated knowing that Smarter Balanced weighs organization and content more heavily than grammar and punctuation. And she said it was valuable to know that errors in punctuation and grammar should be compared to the total amount of writing to determine how pervasive they are.
“A lot of this (training) was designed for professional development,” Grove said. “We want them to learn about the tests…and to work on preparing their kids for performance tasks and creating their own.”