Laurie Udesky/EdSource Today

Students taking Smarter Balanced practice tests at Bayshore Elementary School in Daly City.

With California officials warning that scores on Smarter Balanced tests will likely be lower than those on previous standardized assessments, many teachers want to know how they can better prepare students.

To help demystify the Common Core-aligned tests, the Educational Testing Service, or ETS, is providing several scoring training workshops for teachers this month.

The first Summer Scoring Training for Educators in Sacramento attracted more than 100 teachers and district administrators interested in learning how portions of the tests that are not multiple-choice are hand­-scored.

“A lot of people want this information to take back to their schools,” said Jeff Grove, ETS mathematics assessment director. “A lot of people don’t realize how much is involved.”

The training focused on “performance tasks” students were asked to complete in mathematics and English language arts. These tasks involve classroom instruction and generally require longer responses than questions that are scored by computer.

For example, a math task could involve a scenario that requires students to make change, figure out the volume of an object, interpret a graph or table, or make calculations based on a word problem. An English language arts task could require students to read three passages about a certain topic, then write a short essay or story based partially on the sources provided.

“STAR tests didn’t have that,” Grove said, referring to the state’s previous standardized tests. “This is very new for teachers and students, so it’s important.”

The training included PowerPoint presentations, scoring rubrics and samples of real student answers annotated with explanations about why they received specific scores.

Although districts are gearing up for the release of Smarter Balanced scores from the first tests given statewide last spring, ETS representatives said they hoped the training would prompt more districts to use interim assessments that can be administered during the year. These optional, shorter tests are scored by district teachers and can help educators and their students to practice test­-taking strategies and gain a better understanding of what differentiates high and low scores.

Some teachers and administrators who attended the training said in later phone interviews that they planned to share what they learned with others in their districts to give educators a better sense of elements that are heavily weighted.

“I really wanted to find out across California samples of some of the answers students gave so I’ll know how to improve my teaching of writing,” said Alanna Butterworth, a 3rd­-grade teacher at Camellia Basic Elementary School in the Sacramento City district. “What I took away is that the kids really need to know keyboarding and writing.”

The Sacramento district administered interim tests during the past school year and trained teachers including Butterworth to score them according to Smarter Balanced rubrics, she said. The ETS training workshop, Butterworth said, confirmed that she was scoring correctly by adhering strictly to the rubrics.

According to the rubrics, “effective” elaboration and development merits a 4, “adequate” elaboration and development rates a 3, “uneven, cursory elaboration and development” should get a 2, while “minimal elaboration and few or no details” should receive a 1. Those who don’t understand the content, don’t answer or answer in gibberish receive a 0, Grove said.

Butterworth said some high school teachers in her group felt the rubrics were a bit rigid.

“There was a great deal of disagreement when we tried to come to a consensus about whether this (sample answer) was a 3 or a 4” she said. “High school teachers were questioning, ‘What is the difference between adequate and effective?’”

However, Michelle Campbell, who teaches 8th ­grade English at Katherine L. Albiani Middle School in Elk Grove, said her group – which included both elementary and secondary teachers – ­thought the rubrics were very clear.

“Once we got through our first couple (of samples) and talked about them, we were pretty well calibrated,” she said. “We went back to the annotations to see what made a 4 a 4 or a 3 a 3. We thought it was consistent and we really liked that.”

The Elk Grove Unified School District sent seven teachers to the training, along with two principals and a vice principal. The nearby San Juan Unified School District sent research specialist Lisa Campbell, a former middle school math teacher, since teachers were unavailable due to other professional development.

San Juan used interim assessments last year with selected teachers who were willing to try them, Lisa Campbell said. This year, she said the district plans to train one teacher from each school to score the tests, with the expectation that they will train colleagues at their sites according to the “training of trainers” model.

“We’re scheduling trainings where teachers come and we’re going to give them one of the performance tasks and they will all come together and grade it,” she said. “We want teachers to feel free to use (the interim assessments) so they can use the data.”

ETS did not allow EdSource or teachers who attended the training to reveal the performance tasks or questions for security reasons.

The math performance tasks included six questions based on a common scenario or word problem. In some cases, students were asked to use answers from one question to figure out answers to subsequent questions.

If a student makes a mathematical error in answer to one question, then uses that incorrect answer to respond to another question that is reasoned correctly, the student will get full credit for the subsequent question, Grove said.

English performance tasks included “full write” items worth 10 points and “research short text” items worth 2 points. Full write answers involved narrative, opinion/argumentative, and informative/explanatory writing. Research short text items required short responses to questions that asked students to justify their answers and cite their sources.

After seeing how tests were scored, Lisa Campbell said teachers might tweak their instruction by insisting on source citations in English language arts lessons and on clear explanations of calculations in math.

Last year, Butterworth said her 8-­year-­old students really needed to practice finding the apostrophe key and holding down the shift key to make capital letters and question marks. For math problems, they had trouble finding symbol keys and clicking and dragging items from one place to another on their computer screens, she said.

Butterworth said the Smarter Balanced description of “elaboration” would help her to explain it to students. It includes “relevant details, dialogue, and/or description,” also known as the “three Ds in narrative writing.”

Michelle Campbell said the training would inform her practice.

“I think I’m a better teacher when I know how students are assessed,” she said. “For me, it validates the things I’m already doing. I’m on the right track.”

It was helpful, she said, to understand the individual elements that made up the 10­-point scores. For example, she appreciated knowing that Smarter Balanced weighs organization and content more heavily than grammar and punctuation. And she said it was valuable to know that errors in punctuation and grammar should be compared to the total amount of writing to determine how pervasive they are.

“A lot of this (training) was designed for professional development,” Grove said. “We want them to learn about the tests…and to work on preparing their kids for performance tasks and creating their own.”


Comments

Leave a Comment

Your email address will not be published. Required fields are marked *


Comments Policy

The goal of the comments section on EdSource is to facilitate thoughtful conversation about content published on our website. Click here for EdSource's Comments Policy.

Expand Comments
Collapse Comments
  1. SD Parent 9 months ago9 months ago

    I appreciate teachers trying to understand what the Common Core standards mean, based on the assessments, but this smacks of trying to "teach to the test." I have to agree with Gary on this one: this test is only valuable if we, as society and the future employers, genuinely believe that what is being tested is what we want our future leaders and employees to learn. (For example, I don't believe that … Read More

    I appreciate teachers trying to understand what the Common Core standards mean, based on the assessments, but this smacks of trying to “teach to the test.” I have to agree with Gary on this one: this test is only valuable if we, as society and the future employers, genuinely believe that what is being tested is what we want our future leaders and employees to learn. (For example, I don’t believe that a student needs to explain their logic for reaching a conclusion to a math problem with an essay or needs to show how to solve it three different ways because, in general, if one can answer a word problem correctly, then one has clearly learned to use logic to apply the math concepts, which is what is truly relevant.)

  2. Todd Maddison 9 months ago9 months ago

    This is exactly what should be happening – standardized testing driving teachers to investigate how they can do a better job to teach their kids the things that the test will be measuring. In private industry, if you’re a salesperson whose metric is “total sales” for a period – and your numbers are low – you’re going to either learn how to be a better salesperson OR fail out of that job, perhaps finding some other … Read More

    This is exactly what should be happening – standardized testing driving teachers to investigate how they can do a better job to teach their kids the things that the test will be measuring.

    In private industry, if you’re a salesperson whose metric is “total sales” for a period – and your numbers are low – you’re going to either learn how to be a better salesperson OR fail out of that job, perhaps finding some other career that you’re better at.

    That’s exactly what should happen with teachers. If you’re not good at teaching kids the things that we want them to learn, find another career.

    That’s how we improve education, just like that’s how we improve almost everything in the world.

    Measure the results, learn from those measurements, apply that learning to the process, then cycle back through the testing again to see if the changes you made worked. Rinse, wash, repeat.

    Can you imagine how much better our education would be if, instead of having teachers get routine 2% raises every year just for warming the seat, instead we put a 10 or 20% bonus on raising their kids test scores…?

    And, on the other side, warned them if they don’t they’re going to be asked to find a new career?

    Imagine that – be good at what you do and get paid more, if you’re not good at what you do be encouraged to do something else. What a concept.

    Replies

    • Gary Ravani 9 months ago9 months ago

      Todd: What you say is true only if the tests are measuring that which students really need to know and measuring in ways that make the educationally valuable to be informative, valid, and reliable. We don't have enough information yet to know if that is true about SBAC or the Common Core that the tests are aligned with. Then, after condition #1 is confirmed, there is the issue concerning whether the testing information is actually … Read More

      Todd:

      What you say is true only if the tests are measuring that which students really need to know and measuring in ways that make the educationally valuable to be informative, valid, and reliable. We don’t have enough information yet to know if that is true about SBAC or the Common Core that the tests are aligned with. Then, after condition #1 is confirmed, there is the issue concerning whether the testing information is actually used to inform instruction, or will it be abused to compare students, teachers, and schools in unfair an unproductive ways as occurred under the previous regime of standards and testing.

      You seem to have the “business perspective”of kids as widgets and teachers as production line workers which is not a model applicable to education. Education involves people and relationships not products.

Template last modified: