Credit: Lillian Mongeau/EdSource Today

The initial release of statewide test scores in California this week will reveal how well schools, districts and groups of students did in classes aligned with the Common Core standards. With two-page reports that will be mailed to their homes, parents will find out how well their children did on the new Smarter Balanced tests on the Common Core. And with their own database of information, teachers will learn how effectively they’ve taught them.

The state is gradually rolling out the Online Reporting System, a web-based tool that will enable teachers and principals to easily analyze their students’ end-of-year test results in more detail than under the previous Standardized Testing and Reporting, or STAR, program, which Smarter Balanced replaced. The new testing system is now known as the California Assessment of Student Progress and Performance, or CAASPP. ETS developed the data system for California and other states in the Smarter Balanced Assessment Consortium.

One of the promises of online testing – and Smarter Balanced in particular – was that teachers would receive preliminary student test results quicker – potentially before the end of the school year for most students – and be able to analyze them at a level of detail that will enable them to spot strengths and gaps in their instruction. The Online Reporting System is designed to be the tool that will allow them to do that.

“Better data than before about students in real time is the goal,” said State Board of Education member Patricia Rucker.

The database is still being phased in; at this point, only district and school site administrators have access to student reports in the Online Reporting System. The California Department of Education and ETS expect to complete permission protocols that will extend the data to the students’  teachers by late October. And ETS is dealing with technical issues involved with reporting the data with the detail that will most help teachers pinpoint areas of curriculum and instruction to focus on. Meanwhile, districts could provide printouts of the results or make them accessible by uploading files through their existing data systems. Data reports for the next round of  Smarter Balanced tests, in the spring of 2016, should be available weeks before summer vacation, under the state’s contract with ETS, according to Keric Ashley, state deputy superintendent.

But, from what they’ve seen so far, district assessment administrators and a teacher who got a run-through liked what they saw. The state Department of Education also provided EdSource a preview of the system.

“Teachers want to improve their practice, so they can’t wait to get hold of the data (to ask), ‘What can we do differently?’” said Susan Green, director of assessment, evaluation and planning for the San Juan Unified School District.

Students will receive a score, between 2000 and 3000 on math and English language arts falling within one of four performance levels. Emily's score for English language arts is slightly in Level 3, indicating she satisfied the overall requirements of the standards. The line encompassing the dot is the margin of error.

Source: California Department of Education

Students will receive a score between 2000 and 3000 on math and English language arts falling within one of four performance levels. Emily’s score for English language arts is slightly in Level 3, indicating she satisfied the overall requirements of the standards. The line encompassing the dot is the margin of error.

Initially, teachers will get electronically the same test information that a parent will receive in the two-page paper report, except that a parent will see one child’s score, while teachers will look at class results. Teachers can then break down the information by various student subgroups, such as English learners. Over time, both current and former teachers will be able to compare scores from previous years.

Parents will receive their child’s four-digit scores in math and English language arts. The scores  fall within one of four levels, from Level 1, “standard not met,” to Level 4, “standard exceeded”; other states are equating Level 3, “standard met,” with “proficient,” but California isn’t using the term.

The reports then break down math and English language arts to their key components, which Common Core calls “claims.” These were called “clusters” under the previous state standards.

There are three for math:

  • Problem solving and data analysis (using tools and strategies to solve real-world problems);
  • Concepts and procedures;
  • Communicating reasoning (the ability to explain conclusions).

There are four “claims” for English language arts:

  • Reading
  • Writing
  • Listening
  • Research/inquiry

There won’t be a student score for each component; instead the reports will say whether the student was above, below or at/near the standard for each. This information is useful to teachers, said Julie Steiger, an 18-year teacher who teaches 4th grade at Mariemont Elementary in San Juan Unified. “If you have a group of students who scored high in a claim area, you can focus instead on another area.”

Green cited an instance this month where the information on math claims will change instruction. Teachers at a high-performing San Juan elementary school noticed scores lagged on problem solving and communicating reasoning compared with procedures.

“They realized that they hadn’t been asking deeper questions” involved in mathematical reasoning, Green said. “They’ve decided to build into their curriculum next winter two complex assessments, requiring students to apply their knowledge to solve a problem and then explain their answers. Watching teachers learn and then make changes to instruction this year has been very exciting.”

Data at the claims level is comparable to what parents and teachers received from STAR reports on the California Standards Tests, although Smarter Balanced says that claims emphasize students’ ability to apply knowledge covered by the standards.

The biggest advantage for teachers is that the online system will drill down to the next layer of detail, breaking each claim into multiple components that more closely match the curriculum that teachers teach. These elements are called “targets.” For 5th-grade reading for example, teachers will see, among other targets, how well students:

  • Summarized central ideas, key events, procedures and topics;
  • Identified or interpreted figurative language, like metaphors;
  • Used supporting evidence to support interpretations of information.

Individual students won’t receive scores on targets. They will be statistically valid, depending on the numbers of students tested, only at the class or grade level, Ashley said. ETS should complete the target breakdowns sometime this winter, he said.

By matching targets with standards, teachers could identify strengths and weaknesses in the curriculum and areas of instruction that need work. They could identify subgroups of children in a school that need extra help in particular areas. Scheduling is more time-consuming in high schools, but some elementary schools assign students to the next grade by the end of the school year. Green envisioned the students’ current and next year’s teachers going over data together, to plan for individuals or groups of students.

Perhaps a student identified as gifted did poorly on the Smarter Balanced tests. If assigned to her class, Steiger said, she would come up with a motivational strategy to see that the student starts the year primed to learn.

“The target level is what we are really excited about,” Green said. The usefulness of the data  pinpointing weaknesses and strengths in curriculum and instruction “will be dramatic,” she said.

Timing will be critical. Under ETS’ contract with the state for next year, districts will start receiving individual student results three to six weeks after completing math or English arts tests. They won’t all come at once. Districts with earlier test dates will get theirs first. And the district and school totals are only preliminary; final results won’t be released by the state until August – at least a few weeks earlier than the initial results from this year. But they will be useful for instructional purposes, and for the first time, teachers could receive the data before the end of the school year, depending on when districts administered the tests.

Since STAR was paper-based, teachers had to rely on districts’ central offices to upload CDs of information and provide reports they requested. Teachers should have easier and fuller access to their students’ data under the new system, assuming districts make it available to them. Rucker, a former teacher who is now a lobbyist for the California Teachers Association, is concerned that districts that are used to controlling who gets to see data and when will restrict access. She is optimistic that they will grant it.

Ashley agreed. “It is our expectation that more than in the past, every teacher should have access to make real-time decisions.”


Comments

Leave a Comment

Your email address will not be published. Required fields are marked *


Comments Policy

The goal of the comments section on EdSource is to facilitate thoughtful conversation about content published on our website. Click here for EdSource's Comments Policy.

Expand Comments
Collapse Comments
  1. Adam Ebrahim 9 months ago9 months ago

    I like the Smarter Balanced Assessments. I have spent a great deal of time over the past two years training teachers in SBAC states to write classroom-based assessments in English Language Arts that match the level of rigor of the new assessments. I think the assessment have the potential to help move healthy/inclusive education systems further toward an overall “Claim” of College & Career Readiness. With that said, it is important to note that focusing … Read More

    I like the Smarter Balanced Assessments. I have spent a great deal of time over the past two years training teachers in SBAC states to write classroom-based assessments in English Language Arts that match the level of rigor of the new assessments. I think the assessment have the potential to help move healthy/inclusive education systems further toward an overall “Claim” of College & Career Readiness. With that said, it is important to note that focusing instructional and institutional responses on claim and target data as a proxy for the larger scope of CCSS can be problematic. Certainly the data are important, but they also have limitations.

    Specifically, Smarter Balanced assessments do not produce sufficient data to support ELA Claim #3 “Students can employ effective speaking and listening skills for a range of purposes and audiences.” The claim score is constituted by one target (Listen & Interpret) that includes only 2 of 6 Common Core Speaking & Listening Standards (CCRA. SL.2 + SL.3). I imagine this is why the Claim appears only as “Listening” on student score reports.

    The 4 Speaking & Listening standards omitted are difficult to test in a large-scale standardized way but are absolutely essential to actual College & Career Readiness–effective group collaboration, presentation development/delivery, strategic use of digital media, adapting speech to a variety of contexts. These standards are also essential in supporting English language development, promoting the nested vision of the ELA/ELD Framework, and where many of the 21st Century skills live (along with omitted CCRA.W.6 “Use technology, including the Internet, to produce and publish writing and to interact and collaborate with others.”)

    One of the problems with the CST is that we failed come to agreement about what the results of the test meant, so they came to mean everything. I would caution against repeating this mistake. Prior to poring over target-level data, I would advise having conversations about the place of this assessment within the broader scope of signals that lead to decisions in education systems and classrooms. I would advise having conversations about what it measures well and what it measures poorly/not-at-all. The results can be helpful if they are properly qualified and situated alongside classroom-based formative assessments (professional learning focus) that measure the broader scope of skills our students need to be happy and successful people in the 21st Century.

    Reference: http://www.smarterbalanced.org/wordpress/wp-content/uploads/2011/12/ELA-Literacy-Content-Specifications.pdf

    Replies

    • Don 9 months ago9 months ago

      Adam, It is my view that the former assessment results were rarely used to inform changes in instruction for individual students or even for groups of students. It was all for the purpose of compliance.

      • Adam Ebrahim 9 months ago9 months ago

        I'd love to hear more about your perspective. In my experience CST scores were usually followed by pretty onerous prescriptions. Students routed into intervention courses, packets of test prep showing in my room, kids being pulled out of PE to do test prep computer programs, arts courses and cte being cut, mandatory data sessions to identify/focus on kids who could be moved a performance bandwidth followed by mandates to identify them on our seating charts … Read More

        I’d love to hear more about your perspective. In my experience CST scores were usually followed by pretty onerous prescriptions. Students routed into intervention courses, packets of test prep showing in my room, kids being pulled out of PE to do test prep computer programs, arts courses and cte being cut, mandatory data sessions to identify/focus on kids who could be moved a performance bandwidth followed by mandates to identify them on our seating charts and focus on them. Pretty terrible things happened here in Fresno.

        • David B. Cohen 9 months ago9 months ago

          Glaring inequities, and yes, what you describe does sound terrible. My school/district never even shared students' CST results with us, and though I had access to them, I very rarely looked them up (less than 1% of my students in a decade). It wasn't that I didn't care about their learning, but rather that their test scores told me nothing I wanted to know, and since my school and district had strong test scores, no … Read More

          Glaring inequities, and yes, what you describe does sound terrible. My school/district never even shared students’ CST results with us, and though I had access to them, I very rarely looked them up (less than 1% of my students in a decade). It wasn’t that I didn’t care about their learning, but rather that their test scores told me nothing I wanted to know, and since my school and district had strong test scores, no one was pressuring us to raise them collectively. With the new tests, I’m curious enough to look again. In the long run, I can see where broad views of data might help raise some useful lines of inquiry for staff. However, if we can’t see the assessments themselves, I won’t necessarily conclude that the patterns in the data indicate strengths or weaknesses in our instruction. And at the individual level, I would trust my own assessments over SBAC at this point. I’ve heard way too many stories about testing irregularities and variables that are non-standardized, and I know students have good days and bad days, and I know that there’s a fair amount of random chance involved if you only try to assess a certain skill/standard with 1-2 multiple choice questions.

  2. Gary Ravani 9 months ago9 months ago

    There is a conundrum re timing here. Yes, it would be useful for teachers to get test results earlier, but that means giving the tests earlier and not having enough instructional time to cover the tested standards. One of the major problems with the CSTs were that they "potentially" covered the full year's worth of standards, but you only had about 3/4ths of the year to instruct on those standards before the testing window opened. … Read More

    There is a conundrum re timing here. Yes, it would be useful for teachers to get test results earlier, but that means giving the tests earlier and not having enough instructional time to cover the tested standards. One of the major problems with the CSTs were that they “potentially” covered the full year’s worth of standards, but you only had about 3/4ths of the year to instruct on those standards before the testing window opened. I’m not sure the goal of using these summative assessments to inform instruction is going to work…ever. The proposed interim tests might be helpful, but that won’t get reported. And there’s only so much adjusting of curriculum and potential reteaching that can be accomplished.

    Replies

    • el 9 months ago9 months ago

      There really shouldn't be a conundrum about timing. I don't think anyone wants them given sooner in the year; I think most people are confused about why it is that it takes all summer to report the data back. I suspect the issue is taking all the score results and then choosing the cutoffs for the different band labels. People think that the score is based on some absolute expectation, but in truth it's edited … Read More

      There really shouldn’t be a conundrum about timing. I don’t think anyone wants them given sooner in the year; I think most people are confused about why it is that it takes all summer to report the data back. I suspect the issue is taking all the score results and then choosing the cutoffs for the different band labels. People think that the score is based on some absolute expectation, but in truth it’s edited based on how many kids got each question correct and what the curve of absolute scores looks like.

      The variation in when the test is given during the year should be expected to introduce significant error and differences as well. That would be a little interesting… a graph of scores versus date of exam.

  3. Barbara 9 months ago9 months ago

    What did I learn? That the Common Core is a bore and I refuse to teach to it. Parents: Opt your kids out. Schools: Bring back vocational ed, honors, AP and internships. Use authentic assessment so we can see kids real talents. Stop setting schools up to fail.

    Teachers: Stop buying into the corporate education model.

  4. Todd Maddison 9 months ago9 months ago

    Why is this data not going to be available to the public? I can understand why individual student results would not be viewable by all, but why can't I see the results for a specific school district, specific school, and specific teacher in that school? I"m paying the salaries of "all of the above", why can't I get visibility into what I'm paying them for? Read More

    Why is this data not going to be available to the public? I can understand why individual student results would not be viewable by all, but why can’t I see the results for a specific school district, specific school, and specific teacher in that school?

    I”m paying the salaries of “all of the above”, why can’t I get visibility into what I’m paying them for?

    Replies

    • Gary Ravani 9 months ago9 months ago

      Todd:

      We have now had over a decade of looking at test scores down to the granular level. That model has done nothing to improve education. There was data available on states, on districts, and on individual teachers. (See the LA Times.) As someone once said: “Data is not knowledge. And even if it was, knowledge is not wisdom.” The “data driven” education model has proven to be spectacularly lacking in wisdom.

      • Todd Maddison 9 months ago9 months ago

        Name a school system administrator, principal, or teacher who has a bonus program based on student achievement. Yes, we do have over a decade of looking at test scores - but that's all we've done. Look at them. If you feel that you can actually improve a process by simply looking at the results and doing nothing, I'm guessing you're not a science teacher, perhaps? In the real world, process improvement is done by testing the output … Read More

        Name a school system administrator, principal, or teacher who has a bonus program based on student achievement.

        Yes, we do have over a decade of looking at test scores – but that’s all we’ve done. Look at them.

        If you feel that you can actually improve a process by simply looking at the results and doing nothing, I’m guessing you’re not a science teacher, perhaps?

        In the real world, process improvement is done by testing the output of a process, making adjustments, testing again, finding out what works, refining the process, testing again, making adjustments, etc..

        The fact is that we’ve never done anything with the test results we have that would result in improvement, hence no improvement.

        • el 9 months ago9 months ago

          It is the students that achieve or don’t, so why not give the merit payments to them?

          • TheMorrigan 9 months ago9 months ago

            You are right, el. Most people forget that students are the ones who do the learning and not the teachers. Teachers do not fill their kids heads with knowledge. The teacher bonus system, which has been tried again and again, which I call a repetitive and needlessly resurrected zombie for over 80 years, eventually follows the paths of cannibalizing itself or shambling to dust. The meta-research on that is very clear. However, student bonus systems … Read More

            You are right, el. Most people forget that students are the ones who do the learning and not the teachers. Teachers do not fill their kids heads with knowledge.

            The teacher bonus system, which has been tried again and again, which I call a repetitive and needlessly resurrected zombie for over 80 years, eventually follows the paths of cannibalizing itself or shambling to dust. The meta-research on that is very clear. However, student bonus systems have a track record that is promising, yet it never seems to get the societal or political impetus that it justly deserves.

      • Dawn Urbanek 9 months ago9 months ago

        Gary - After pouring through data over the last couple of days, I have to disagree with you. I looked at data today which showed enrollment for the Capistrano Unified School District. When you read the Board Agendas parents are told enrollment is declining. Enrollment according to data quest is increasing and is stated to be around 54,000. When I look at the budget documents we get ADA based on enrollment of 48,000 students. That would … Read More

        Gary –

        After pouring through data over the last couple of days, I have to disagree with you. I looked at data today which showed enrollment for the Capistrano Unified School District. When you read the Board Agendas parents are told enrollment is declining. Enrollment according to data quest is increasing and is stated to be around 54,000. When I look at the budget documents we get ADA based on enrollment of 48,000 students. That would equate to about $44 million dollars at $7,667 per student. So why would ADA differ so much from actual enrollment.

        • el 9 months ago9 months ago

          Dawn, ADA is paid based on actual attendance in class, per day. The ADA/enrollment numbers you cite would suggest that your district has a 89% attendance rate, which would be low (ie your average student counts as chronically absent). It's more likely that enrollment varies quite a bit over the year - enrollment may have been higher at the beginning and lower at the end. The school gets money each day a child attends; the … Read More

          Dawn, ADA is paid based on actual attendance in class, per day. The ADA/enrollment numbers you cite would suggest that your district has a 89% attendance rate, which would be low (ie your average student counts as chronically absent). It’s more likely that enrollment varies quite a bit over the year – enrollment may have been higher at the beginning and lower at the end. The school gets money each day a child attends; the school gets no money if the child does not attend, even if the absence is excused.

          • Dawn Urbanek 9 months ago9 months ago

            Thank you El- That is what I thought. It makes no sense that our District is at 89% when every other District in our area is around 95%. May I ask who calculates that number? We have many schools that are impacted and not enough seats for students- is the District making an error in counting students? We can't have 5,500 students that are perpetually absent- not the make-up of this District. Any other suggestions? … Read More

            Thank you El- That is what I thought. It makes no sense that our District is at 89% when every other District in our area is around 95%. May I ask who calculates that number? We have many schools that are impacted and not enough seats for students- is the District making an error in counting students? We can’t have 5,500 students that are perpetually absent- not the make-up of this District. Any other suggestions? Is there any incentive to underreport ADA? I would not think so- so I wanted to ask.

            Thanks you for your response.

            • el 9 months ago9 months ago

              Dawn, it looks like you can explore this data some yourself. Go here: http://dq.cde.ca.gov/dataquest/ You can get an enrollment report for your district or school level. Fiddle with that, see what you get. I'm not sure if you can capture enrollment changes during the year with this report. There's no incentive to underreport ADA - districts are paid based on that. The district financials will have ADA numbers. The thing is that all the numbers move. It's completely … Read More

              Dawn, it looks like you can explore this data some yourself.

              Go here: http://dq.cde.ca.gov/dataquest/

              You can get an enrollment report for your district or school level. Fiddle with that, see what you get. I’m not sure if you can capture enrollment changes during the year with this report.

              There’s no incentive to underreport ADA – districts are paid based on that.

              The district financials will have ADA numbers.

              The thing is that all the numbers move. It’s completely possible that your district could start the year at 54,000 and end it at 50,000 students, for example, and it might have peaked at 56,000 or bottomed out at 48,000 in between. Your ADA for day 10 of the school year might be 52,000 and your ADA for day 170 might be 48,000, so a one number report will be the average. It would be nice to be able to find attendance percentage per day of school, for example, because these numbers will be quite different depending upon each day’s movement in enrollment.

Template last modified: