Credit: Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action

California’s current approach to presenting and analyzing education data obscures tremendous successes and failures. As a result, teachers and principals miss out on some of the most meaningful learning the data offers.

David Wakelyn

News this week about the 2018-19 Smarter Balanced test results concluded that growth is slow and “scores are stagnant.” California, like most states, compares students in this year’s grade to the same grade the previous year.In 2019, 48.5 percent of 3rd-grade students met ELA standards, up a meager 0.3 percent from the previous year.

Instead, if schools follow the same groups of students over time the results are very different: 48.2 percent of 3rd-graders were reading at or above state standards in 2017-18. As they moved into 4th grade, 54 percent of them were reading at standards, a gain of nearly 6 percent.

If schools begin to track achievement for a class of students over time, they can generate new insights that shape teaching and learning. I rearranged the publicly available K-12 data to create groups of California students based on when their classes will eventually graduate from high school. Last year’s 3rd-graders are in the Class of 2028, 4th-graders in the Class of 2027 and so on.

Each class averages around 458,000 students at the start of 3rd grade and fluctuates in size by 0.4 percent from year to year. While the size of classes can rise and fall within schools (as families move to different neighborhoods), they remain stable in most districts and the state as a whole.

What I found surprised me.

In English Language Arts, in the four classes that have taken Smarter Balanced tests since entering 3rd grade, 123,000 more students are meeting and exceeding standards. The class of 2025 saw a dip last year, though its trajectory is one of long-term improvement.

In mathematics, the data reveal a more concerning story: 103,000 fewer students are meeting or exceeding standards in 4th, 5th, 6th and 7th grades than they were as 3rd-graders.

California is unable to see these trends and pose basic questions that other states can easily answer, because it has not tracked individual students’ growth as part of its school accountability system.

The state does include a limited “change over time” feature on its CAASPP website. But because it only displays percentages, it doesn’t show the change in the number of students performing at each level. Looking at this new feature, you’d have no way of knowing that 40,000 fewer 5th-graders are meeting math standards than they were as 3rd-graders.

With a few exceptions, California’s trend of a rise in ELA proficiency and a slide in math holds across all 11 states publicly reporting Smarter Balanced achievement data. But there is one important divergence: California, Connecticut and Idaho have been able to reverse the decline in math. They showed a “6th grade bounce.” The classes of 2022 and 2023 showed it too and then plateaued as students moved into 8th grade.

These trends are evident in national data as well. If you follow cohorts of 4th-graders who take the National Assessment of Education Progress (NAEP) and see how they achieve four years later, math declines and reading bumps up slightly. But the NAEP is only administered every two years to nationally representative samples of students in grades 4 and 8.

The advantage of the Smarter Balanced data is we can see these changes year to year and begin to understand how to work with each class of students differently.

In order to meet California’s employment needs, 103,000 more — not fewer — students need to be meeting and exceeding the state’s math standards. One of the lead writers of the Common Core standards in math, Jason Zimba, wrote in 2014, “Arithmetic is like the handle of a wrench: grasped firmly it gives you leverage. Students who don’t have the right foundation in arithmetic will struggle in later grades.”

What should California do? First, we need to generate and lock in feedback from expert teachers. The state, or a group of concerned districts, should convene elementary teachers at each grade level and begin to generate hypotheses about what the contributing factors to the success and failure might be.

Master teachers can explore what needs to be different and what’s going well. Is pacing off: Are teachers moving too fast or too slow through certain math topics? Are teachers doing something valuable with the way they organize small reading groups?

The regional conferences of the California Mathematics Council provide one venue for these inquiries. The initial answers to these questions don’t need to determine definitive causes, but can serve as the start of an ongoing inquiry to guide colleagues in the 11 states that administer the Smarter Balanced tests.

Second, we need to change the way we see and analyze the data. The California School Dashboard — which measures performance at a school, district and state level — needs to follow groups of students over time, showing the trends across all performance categories.

The State Board of Education is investigating whether to adopt a student growth model and will look to an advisory committee for a recommendation, but it will not decide on adoption until late 2020, at the earliest. It should accelerate this timeline.

The Math-in-Common network, a consortium of 10 California districts, has been helping math teachers navigate roadblocks to improving their instruction. While these efforts are “grounded in theories of improvement”, their reports make it difficult to see the magnitude of the problem that an analysis of individual students’ growth reveals: that tens of thousands of students who once were meeting or exceeding state standards no longer are. Two of their districts — Long Beach and Garden Grove Unified have shown less of a slide in math and are making the largest gains in English.

Finally, we need to build case studies of individual student success and failure. Analyze a group of students who started out meeting standards in 3rd grade and determine why they have slipped in 5th and 6th grades, assembling a complete picture of their academic and social foundations.

Similarly, we need to review a group of students who initially weren’t meeting standards and have grown to do so. Research in organizational leadership emphasizes the need to study our successes so we can replicate and build on them.

Learning is about understanding why things happen. For all the rhetoric around “continuous improvement” in California, it’s hard to learn much of anything unless we’re studying both our successes and failures using analytical tools that are available but not used.

•••

David Wakelyn is a consultant at Union Square Learning, a nonprofit that works with school districts and charter schools to improve instruction. He previously was on the team at the National Governors Association that developed Common Core State Standards

The opinions expressed in this commentary represent those of the author. If you would like to submit a commentary, please review our guidelines and contact us.

We need your help ...

Unlike many news outlets, EdSource does not secure its content behind a paywall. We believe that informing the largest possible audience about what is working in education — and what isn't — is far more important.

Once a year, however, we ask our readers to contribute as generously as they can so that we can do justice to reporting on a topic as vast and complex as California's education system — from early education to postsecondary success.

Thanks to support from several philanthropic foundations, EdSource is participating in NewsMatch. As a result, your tax-deductible gift to EdSource will be worth three times as much to us — and allow us to do more hard hitting, high-impact reporting that makes a difference. Don’t wait. Please make a contribution now.

Share Article

Comments (5)

Leave a Reply to Dr. Bill Conrad

Your email address will not be published. Required fields are marked * *

Comments Policy

We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.

  1. Eugene Park 2 months ago2 months ago

    Doug, you are certainly right about the issue of student mobility when using aggregated data to conduct a longitudinal analysis. The public data files are limited, but through creative data visualization techniques you can learn a lot about a school, district, or state. I’m constantly fine-tuning this, but I have made the state and county-level 2018-19 CAASPP analysis available on our website. Let me know what you think. https://www.parseceducation.com/pages/2018-19-state-and-county-caaspp-presentations

    Replies

    • Doug McRae 2 months ago2 months ago

      Eugene:: Nice set of graphical reports. The only comment I would have might be to add a red * for all trend or prediction points if the n-count is less than (say) 500 for statewide and county graphs, including subgroup graphs. Also, don't display points with n-counts less than 100, treat those points the same as n-count 10 or less for numerics displayed with n-counts nearby. If you do graphs for districts and … Read More

      Eugene:: Nice set of graphical reports. The only comment I would have might be to add a red * for all trend or prediction points if the n-count is less than (say) 500 for statewide and county graphs, including subgroup graphs. Also, don’t display points with n-counts less than 100, treat those points the same as n-count 10 or less for numerics displayed with n-counts nearby. If you do graphs for districts and schools including subgroups, then follow the same n-count business rules. Doug

  2. Doug McRae 2 months ago2 months ago

    Looking at same student scores over time (longitudinal analysis), rather than different cohorts of same grade students over time (cross-sectional analysis) is a worthwhile thing to do. But one needs to carefully address two elements that can undermine comparisons of any given group over time. The first element is potentially cherry-picking the full database for only students who have scores for the same cohort (school level, or district level, or subgroup assignment, or statewide data) for … Read More

    Looking at same student scores over time (longitudinal analysis), rather than different cohorts of same grade students over time (cross-sectional analysis) is a worthwhile thing to do. But one needs to carefully address two elements that can undermine comparisons of any given group over time.

    The first element is potentially cherry-picking the full database for only students who have scores for the same cohort (school level, or district level, or subgroup assignment, or statewide data) for both grade levels involved. Some data sets will have large percentages of mobile students who have scores for only one of the two grade levels, and thus will be eliminated from the longitudinal analyses. Such efforts provide a biased view of gains or losses based on cherry-picked “stable” students only.

    The second element is that longitudinal analyses involve measurement of two different sets of content standards (for example, grade 5 vs grade 6 math) that yield less credible results than cross-sectional gain scores over time for two successive cohorts of students. The differences in composition of cohorts over time can many times be less than the differences in the content standards over grades.

    The mobility constraint is likely the more important element to address for longitudinal analyses. For an extreme example, I recollect one classroom of students at a school that served many families of farm workers, many English Learners for recent immigrants from Mexico. The classroom started with 30 students in the fall, had 30 students the following spring, but only 7 students enrolled for the full year. About half the 30 beginning students transferred to other schools during the school year, replaced by new students. About a quarter of the original 30 students left in November to follow the crops to Arizona, but returned to the same classroom the following March or April. Conducting longitudinal analyses for only 25 percent the students taught the whole year is not a reasonable thing to do. Other examples of high mobility student populations include schools with many military families and even schools near universities with concentrations of parents pursuing graduate or professional degrees.

    The above considerations apply to student growth models now being considered by California’s State Board of Education. Student growth models are high profile shiny objects right now, but all prospective models have considerable downsides for schools and/or subgroups with substantial mobility. Such models also have to address the issue of differential academic content standards over grade levels.

    Looking at achievement for a single cohort of students over time (as contrasted to successive cohorts of students over time at the same grade level), can be useful. But, it is not uniformly the best analysis all the time.

  3. Ann 2 months ago2 months ago

    "...convene elementary teachers at each grade level and begin to generate hypotheses about what the contributing factors to the success and failure might be...can serve as the start of an ongoing inquiry to guide colleagues in the 11 states that administer the Smarter Balanced tests." Seriously? Gather and generate hypotheses to begin an inquiry. This is his solution to a failing system 5 years in. That is the elementary years of hundreds of thousands … Read More

    “…convene elementary teachers at each grade level and begin to generate hypotheses about what the contributing factors to the success and failure might be…can serve as the start of an ongoing inquiry to guide colleagues in the 11 states that administer the Smarter Balanced tests.” Seriously? Gather and generate hypotheses to begin an inquiry. This is his solution to a failing system 5 years in. That is the elementary years of hundreds of thousands of kids, not to be repeated or remediated. Perhaps we should look to those who aren’t responsible for the failing system, instead.

  4. Dr. Bill Conrad 2 months ago2 months ago

    We really don't need to continue to convene "expert" teachers to find the solution to woeful academic performance by students. There is abundant cognitive and empirical research showing that it is the alignment of high quality professional practices along with high quality curriculum that will get academic results for students. The solution to that problem is to transform the pathetic colleges of education so that they can recruit the finest and train them … Read More

    We really don’t need to continue to convene “expert” teachers to find the solution to woeful academic performance by students. There is abundant cognitive and empirical research showing that it is the alignment of high quality professional practices along with high quality curriculum that will get academic results for students.

    The solution to that problem is to transform the pathetic colleges of education so that they can recruit the finest and train them well. The teachers should then be part of an ongoing career ladder that begins with novice, goes to apprentice, to journeymen, to Master teachers. Administrators including principals should be derived from the Master teacher level for temporary assignment as administrators . They should spend 85% of their time in classrooms modelling high quality practices, coaching, and evaluating. The goal should be to become a Master teacher. Master teachers should be paid 6-figure salaries with a stipend for administrative work.

    It would be better to engage in this transformation sooner rather than later. WE continue to grope around in the fog of education for solutions. At least it keeps us busy!