Don’t use the SAT or ACT to undermine education accountability

The May 26, 2019, commentary in EdSource has some fundamental inaccuracies that negate the authors’ argument that the SAT shouldn’t be used for accountability. The biggest problem is that the authors discuss the old SAT and ignore the new SAT, which was first introduced to students in March 2016.

First, the new SAT was redesigned to be an achievement test. It measures what students are learning in school that is essential for college and career readiness. The thousands of teachers and curriculum specialists across the nation who participated in the redesign process or who are involved in the test development process have examined the test specifications for the new SAT (go here) and concluded that it measures skills and concepts essential for college and career readiness. The skills and concepts measured by the new SAT were identified by empirical research. (This evidence is posted at the above website as well.) That’s why there is a very close alignment of the SAT to the California state standards and standards of those states committed to college/career readiness.

Simply put, all vestiges of the old aptitude approach used to develop the early SAT 100 years ago have been eliminated. Any claims that the redesigned SAT is not an achievement test are false.

The article claims that the SAT’s technical design ensures that the distribution of student test scores is virtually identical from year to year. That is not true. The redesigned SAT is developed and assembled to measure the content and skills in the test specifications. That is the highest priority. Contrary to the authors’ claim, it is not assembled primarily on the basis of statistics to either maximize differentiation among students or fit them on a bell-shaped curve. Because there is no artificially imposed curve used to score the test, it would be possible, theoretically, for most or all students to earn high scores and to meet and exceed the College Board’s College Readiness Benchmarks (go here).

Second, the only national validity study using national data on the new SAT has just been released by College Board (go here). Studies of new SAT scores in California show that the SAT and high school GPA are independent and equally strong predictors of first-year performance at the University of California (UC), and when these measures are combined, there is a 16% boost in predictive power over high school GPA alone. Moreover, the UC study shows that the SAT adds additional predictive value over high school GPA that, if ignored, would mask students who would likely be successful at UC. That’s only one of the several reasons the claims made by the authors about the SAT being a weaker predictor of college performance than high school GPA are inaccurate.

Third, another troubling error made by the authors involves the College Board’s Environmental Context Dashboard (ECD), which they claim confirms a limitation of college admissions tests. This claim completely misrepresents what the ECD tool offers and why it’s being piloted with colleges. The ECD presents environmental factors that contextualize the high school and home neighborhood of students. These factors affect their scores on any achievement test, including the Smarter Balanced Test. Environmental and school factors clearly impact student achievement and success and add important insights about the context of students’ academic achievements that colleges and universities may want to consider when reviewing applicants.

Finally, the authors claim that the SAT is “calibrated to rank order students from the 0th percentile to the 99th.” Again, this is false. This is not the way the new SAT is developed, designed, or assembled as an achievement test. The SAT measures student achievement just as a thermometer measures temperature. When we see differences in test scores, those differences are not caused by the test: what we are seeing is how well students were instructed and how well they learned essential content and skills in high school. While College Board has many strong disagreements with the authors, the authors would likely agree with us that achievement tests like the SAT have exposed inequities in the quality of instruction in our schools for decades.

Addressing that injustice is one of the greatest challenges we face as a nation, and one of the most important.

Thanks for pointing out the fundamentally different purposes (aptitude vs mastery) of the SAT/ACT and the Smarter Balanced Tests. Teachers and schools can’t do anything about aptitude. Mastery is what they work on, and so that should be what they measure. But I am uncomfortable when well-meaning educators try to avoid talking about bell curves (which are a fact of life) because they reflect negatively on the measured aptitude of Black and (to a lesser extent) Hispanic students. That’s not a subject to avoid but to embrace.

I have tutored more than a hundred Black high school students over 20 years, and there is no difference that I can tell between the distribution of their aptitude as compared to that of other racial groups. Black and Hispanic students do perform worse on aptitude tests like the SAT and ACT, but it’s because, as a group, they lack self-confidence. And they lack self-confidence because, as a society, we have yet to come to terms with the widespread marginalization they endure in a still largely racist society.

Self-confidence is extremely important when you tackle a math or language problem, and nothing undermines a person’s self-confidence more than being repeatedly told (in subtle and not-so-subtle ways) that, “unfortunately and through no fault of your own,” you are deficient.

SAT and ACT scores should definitely not be used as an accountability measure as they are not closely aligned with the Math and ELA standards that schools are expected to ensure that their students achieve. The SBAC is a better more and standards-based measure.

11th grade scores on the SBAC are embarrassingly low compared with Grades 3-8 performance. Check out results for Silicon Valley School Districts at http://sipbigpicture.com District Administrartors blame the students for not taking an interest in these assessments rather than looking in the mirror and understanding their role as the grownups to make sure that students are prepared to take the test and do their best work.

We should not be redirecting our accountability to the SAT and ACT where many affluent students take prep courses and are thus their results are not a true reflection of how well the system is preparing students for success in college and career readiness. The state tried to provide cover for the school districts by initially keeping the 11th grade SBAC scores off of the Academic Dashboard and then using a convoluted scale score averaging system that redistributes excess scale score points from high performing to low performing students creating a biased view of actual performance.

It is time for school districts to get their acts together and begin taking the SBAC seriously as a measure of student achievement of ELA and Math Standards. Their attitude should be bring it on rather than trying to weasel out of their responsibility to successfully prepare their students for success on the SBAC.