To shine a brighter light on academic disparities, the six California districts known as the CORE districts have tracked test results for much smaller student subgroups than the state requires, giving a more complete picture of how some groups – African-American children and students with disabilities, in particular – performed.

CORE (California Office to Reform Education) has reduced the size of student subgroups it uses in its new school accountability system from 100, the minimum the state currently requires, to 20. Reducing the subgroup size for reporting scores on standardized tests succeeded “in making historically underserved student populations visible” and provides schools and districts with critical information to address these students’ needs, according to a new study by Heather Hough, who is directing studies of the CORE districts for the Stanford University-based organization Policy Analysis for California Education, and Joe Witte, a PACE analyst. The State Board of Education should pay attention to CORE’s data since it must decide whether to stick with or change the subgroup size for the school accountability system it’s in the process of creating, the report said.

The districts participating in CORE include three of the state’s four largest districts – Los Angeles, Long Beach and Fresno – along with Santa Ana, San Francisco and Oakland, and enroll about a million of the state’s 6 million students. As a condition for receiving a federal waiver from some of the constraints of the No Child Left Behind Act, the CORE districts designed their own school accountability and improvement system, called the School Quality Improvement Index. The multi-dimensional index, introduced last year, includes indicators, such as rates of chronic student absenteeism and suspensions, that the state board is considering. Another metric, self-reported measures of social and emotional skills, is novel and controversial.

The CORE districts hope the state board will let them continue using their accountability model to identify schools needing improvement instead of the state’s new system, which will go into effect in 2017-18. CORE will make a formal request this fall (see letter to the State Board laying out its reasoning).

CORE Executive Director Rick Miller said the smaller subgroup size is central to its commitment to detect schools’ struggling groups of students who otherwise might go undetected. The PACE study, using scores from last year’s Smarter Balanced math assessment on the Common Core standards at CORE’s 1,153 schools, bore that out.

  • Had the subgroup size been 100, schools would have singled out only 37 percent of African-American students’ test scores; the rest would have been indistinguishable from schools’ total results. The percentage of African-Americans’ scores reported as a subgroup rose to 88 percent with the minimum subgroup size of 20.
  • Only 25 percent of students with disabilities would have been reported as a subgroup if the minimum group size had been 100, but this number was 92 percent with a subgroup size of 20.

Under the CORE index, a school’s lowest-performing ethnic or racial group is included in determining a school’s overall performance. If the subgroup size had been 100, there would have been 70 schools in which African-American students were the lowest-scoring group. With a group size of 20, the number jumped to 333 schools.

States have discretion in setting their subgroup size, and there’s no right or wrong number. Before Congress passed the No Child Left Behind Act, California set 30 students, as long as they made up at least 15 percent of a school’s population, as the minimum subgroup size for reporting results on state standardized tests. But NCLB raised the stakes, with the requirement that all subgroups in a school achieve 100 percent proficiency in math and reading; there were escalating penalties for those schools that failed to meet annual proficiency targets. In response, the state board raised the subgroup reporting size to 100 students in an attempt to reduce the number of subgroups potentially exposing a school to sanctions, according to Miller, who was an assistant state superintendent at the time.

Soon the state may return to 30-student subgroups. Congress ended NCLB, and passed the Every Student Succeeds Act, which transfers oversight of struggling schools to the states. The Local Control Funding Formula, which the state Legislature passed in 2013, emphasizes strategies to improve schools instead of punishing them; it also set 30 as the subgroup size that districts should use in reporting metrics in Local Control and Accountability Plans that districts write laying out district goals and spending actions to achieve them.

Consistent with the funding formula, the California Department of Education will recommend that the state board adopt the 30-student subgroup threshold for the new accountability system that meets both state and federal requirements.

Asked to look at the PACE study of 20-student subgroups, Edward Haertel, an emeritus professor of education at Stanford University and an expert in educational testing and assessment, wrote that while it’s obvious that reducing the group size will increase the number of schools with numerically significant groups, “the magnitude of these effects reported by PACE is striking.” But he cautioned that the smaller the group size, the larger the risk of inaccuracy, which is compounded when using the information for accountability purposes.

But Miller said CORE believes the benefits of a smaller subgroup size outweigh the risk of measurement error – even if the consequences are over-identifying schools needing help.

“What is the harm if the result of a measurement error is to provide more assistance for students? We believe it’s the right thing to do,” he said.

SHARE ARTICLE

Comments (12)

Leave a Comment

Your email address will not be published. Required fields are marked *

Comments Policy

The goal of the comments section on EdSource is to facilitate thoughtful conversation about content published on our website. Click here for EdSource's Comments Policy.

  1. Jim Mordecai 2 years ago2 years ago

    Seems that number of CORE Districts keeps changing. So when is a CORE District a CORE District Mr. Fensterwald? Definition I like is CORE District is a district whose superintendent participates as one of the ten directors of a private corporation calling itself CORE. CORE has an advertised mission to promote reform of California education. But, the work of CORE is the work of a private corporation performed by superintendents employed … Read More

    Seems that number of CORE Districts keeps changing. So when is a CORE District a CORE District Mr. Fensterwald?

    Definition I like is CORE District is a district whose superintendent participates as one of the ten directors of a private corporation calling itself CORE.

    CORE has an advertised mission to promote reform of California education. But, the work of CORE is the work of a private corporation performed by superintendents employed in public school districts.

    The opportunity for this private group to influence policy from within a school districts is great and undermines concept of transparency in school board decision making.

    Fact that the CORE sells its services to the ten school districts with a CORE director serving as superintendent makes conflict of interest unavoidable but unfortunately unreported.

    Replies

    • John Fensterwald 2 years ago2 years ago

      I believe Rick Miller answered your question. CORE is a nonprofit guided by the superintendents whose districts belong to it. The numbers will change as districts join and leave.

  2. navigio 2 years ago2 years ago

    So let me get this straight. CORE defined its own metrics because it thought the state/fed ones were inappropriate. Then it used the argument that its own metrics introduced more extreme flaws as a reason to change the min subgroup size? The lowest performing subgroup is a CORE invention and shouldn't be used as part of what they are arguing here. If anything, this should highlight that that metric might not be such a great … Read More

    So let me get this straight. CORE defined its own metrics because it thought the state/fed ones were inappropriate. Then it used the argument that its own metrics introduced more extreme flaws as a reason to change the min subgroup size? The lowest performing subgroup is a CORE invention and shouldn’t be used as part of what they are arguing here. If anything, this should highlight that that metric might not be such a great idea. If 20 is better than 100, then 5 is better than 20 when the argument is that it increases the number of schools ‘called out’. ‘No harm’, right?
    That said, the number should never have been modified for anything other than statistical soundness in the first place. The fact that it was done explicitly to ‘game the system’ shows exactly how absurd our ‘accountability’ really is.
    Here is a solution for all you bean-counting hand-wringers: report a vector of subgroup sizes, from 1 up to the full subgroup size. Then for each, provide a vector of confidence intervals. Then the public could choose to decide how valid any of this stuff is. (We already do something similar for parent education level.)
    Got a problem with privacy? Then you shouldn’t be saying it doesn’t matter if an error leads to extra help at the expense of exposure.
    Got a problem with the ability of the general public to make such decisions on their own? Decide to teach statistics in public schools, for a change.

    Replies

    • John Fensterwald 2 years ago2 years ago

      navigio: Just a reminder that the story pointed out that the state had a subgroup size of 30 before NCLB, so if there was "gaming" of the system, it was raising the number to 100 to reduce the number of subgroups exposed for failing to meet their Annual Yearly Progress. The state Department of Ed is recommending a return to 30. So, if there's criticism, it would be that the CORE districts may have gone … Read More

      navigio: Just a reminder that the story pointed out that the state had a subgroup size of 30 before NCLB, so if there was “gaming” of the system, it was raising the number to 100 to reduce the number of subgroups exposed for failing to meet their Annual Yearly Progress. The state Department of Ed is recommending a return to 30. So, if there’s criticism, it would be that the CORE districts may have gone too low with 20 — a good issue for statistical analysis, not condemnation.

      • navigio 2 years ago2 years ago

        Correct, John. My statement about gaming the system was exactly in reference to that change to 100.
        CORE can be criticized for many things here, but mostly as it applies to logic. The limit on subgroups is only for public record. Schools have access to every single student’s scores and they are the people who decide who needs and gets intervention.

        • Rick Miller 2 years ago2 years ago

          Thanks Navigo, let me try to weigh in here as well to help answer your questions. First, I should clarify that our continuous improvement school report cards include results for any of the following subgroups where there are 20+ students: racial/ethnic subgroups, socio-economically disadvantaged youth, English Learners, and Students with disabilities. The 20+ threshold allows us to represent the results of as many subgroups as possible while ensuring privacy. PACE chose to highlight … Read More

          Thanks Navigo, let me try to weigh in here as well to help answer your questions. First, I should clarify that our continuous improvement school report cards include results for any of the following subgroups where there are 20+ students: racial/ethnic subgroups, socio-economically disadvantaged youth, English Learners, and Students with disabilities. The 20+ threshold allows us to represent the results of as many subgroups as possible while ensuring privacy. PACE chose to highlight African Americans and Students with Disabilities in their findings as groups that are disproportionately better represented when n sizes are lowered and also because these are groups that are frequently low performing. But to see our full reports with all subgroups you can go here: http://coredistricts.org/indexreports/

          Your suggestion of confidence intervals is interesting. While we do not use them in representing a subgroups performance, we have explored the use of confidence intervals based upon n size in the determination of whether a particular subgroup’s results represent a significant gap. For more, you may want to check out our technical guide: http://coredistricts.org/wp-content/uploads/2016/02/CORE-Index-Technical-Guide-SY-2014-15-updated-2.1.16.pdf
          Thanks again, we appreciate the feedback.

  3. Richard Soto 2 years ago2 years ago

    The title of this article should have been “Districts Turn Spotlight on Struggling African American Students.” Where would I go if my districts had a significant number of struggling Hispanic Students and I wanted information?

    Replies

    • John Fensterwald 2 years ago2 years ago

      There would be a breakdown for Hispanic students as well, there are at least 20 Hispanic students in a school. There are in most CORE district schools. To see the individual school reports, go here and click one of your choice. I picked the first one, for Molly Bakman Elementary in Fresno. The top of page 1 shows the breakdowns by percentage. Page 4 shows the performance by subgroup. Because there are frequently fewer than … Read More

      There would be a breakdown for Hispanic students as well, there are at least 20 Hispanic students in a school. There are in most CORE district schools.
      To see the individual school reports, go here and click one of your choice.
      I picked the first one, for Molly Bakman Elementary in Fresno. The top of page 1 shows the breakdowns by percentage. Page 4 shows the performance by subgroup. Because there are frequently fewer than 100 African American students in many schools, the drop to a subgroup of 20 enables the school to report out their scores.

  4. Jim Mordecai 2 years ago2 years ago

    "What is the harm if results of measurement is to prove more assistance for students?" There is a whole bunch of harm, Mr. Miller. Is it not self-serving for your private organization whom you are CEO and spokesperson for to promote increasing the identification of low scoring groups when your private organization provides at a price solutions to districts with low scoring groups of students? Anyone reading your comments should understand that CORE … Read More

    “What is the harm if results of measurement is to prove more assistance for students?” There is a whole bunch of harm, Mr. Miller.

    Is it not self-serving for your private organization whom you are CEO and spokesperson for to promote increasing the identification of low scoring groups when your private organization provides at a price solutions to districts with low scoring groups of students?

    Anyone reading your comments should understand that CORE has an economic interest in increasing the number of school districts that have identified groups with low scores.

    Also, besides recognizing CORE as a private corporation, readers should also understand that the private/public line is often blurred by CORE spokesperson Mr. Miller.

    Does a CORE school district exist? You would think so in reading this article. But, Oakland School Board for example has never voted to join CORE. However, its Superintendent Antwan Wilson is apparently an ex-officio member of CORE. CORE used to be directed by 10 California superintendents–and since the article names six school districts–I’m guessing CORE is down to six school districts superintendents participating. My point is if a school board does not join CORE–where they would be subordinate to their subordinate–then I don’t think it is correct for Mr. Miller to call a school district a “CORE District.”

    Replies

    • Rick Miller 2 years ago2 years ago

      Jim, thanks for your comment. For the record, I do think it’s important to be clear that CORE is a nonprofit organization. Modeled after teacher PLC’s, CORE, as an organization, simply serves a hub to facilitate meaningful collaboration and learning between our 9 member school districts: Fresno, Garden Grove, Long Beach, Los Angeles, Oakland, Sacramento, San Francisco, Sanger and Santa Ana Unified. Our actions are guided by the superintendents of these school districts, … Read More

      Jim, thanks for your comment. For the record, I do think it’s important to be clear that CORE is a nonprofit organization. Modeled after teacher PLC’s, CORE, as an organization, simply serves a hub to facilitate meaningful collaboration and learning between our 9 member school districts: Fresno, Garden Grove, Long Beach, Los Angeles, Oakland, Sacramento, San Francisco, Sanger and Santa Ana Unified. Our actions are guided by the superintendents of these school districts, who serve as our board of directors.
      A key focus of this work is improving learning and achievement, and providing more equitable learning opportunities for students who historically have been left behind. As the article makes clear, school districts in CORE are gathering and using data in innovative ways to help educators better understand the needs of students and schools, and using that information to develop strategies and practices to improve student learning. Thanks again.

  5. G J 2 years ago2 years ago

    Thanks for this article. There are 10 CORE districts, not 6: Sanger, Clovis, Oakland, Sacramento City, Garden Grove, San Francisco, Santa Ana, Fresno, Long Beach, and Los Angeles.

    Replies

    • John Fensterwald 2 years ago2 years ago

      The CORE website lists nine member districts (not Clovis), only the 6 districts I mentioned received the federal waiver and participated in the initial index.