Credit: Alison Yin for EdSource
Credit: Alison Yin for EdSourceWouldn’t it be nice if all teachers had verifiably excellent instructional materials at their disposal and if every school and school district had a reliable guide as to which instructional materials are effective, before adopting — and paying for — such?
Clearly, it would be nice. And it might actually help improve children’s education — were such labeling true and based on evidence. Yet the people who brought you the mediocre Common Core are now engaged in convincing everyone that textbooks, simply by virtue of being aligned with Common Core, are necessarily also “high quality” and “effective.” They do it without any empirical evidence of them actually being of high quality or effective, while disparaging the quality and effectiveness of textbooks that were proven as truly effective by successful widespread use.
Before 2010, when every state had its own educational standards, some states attempted to provide such guidance by reviewing and creating a list of approved textbooks from which schools and school districts were encouraged — and sometimes forced — to select. Other states left such decisions to schools and school districts. Neither system worked very well. Textbook publishers had significant financial sway over state and local selections, too often overriding the textbooks’ academic merit. Further, the review committees, particularly local ones, often lacked competent content reviewers and input from parents. Publishers added to the confusion by having multiple versions of essentially identical textbooks customized in a minor way for different states. All in all, not a very good situation, even though some highly effective, even if not fashionable, textbooks such as Saxon Math, Singapore Math, or Open Court Reading managed to survive.
The situation has changed since the adoption in 2010 of Common Core standards by almost all of the country. Now most states have kind-of the same standards so, suddenly, rather than competing with each other over each of the 50 states, textbook publishers could focus on competing at a single national level.
In theory this could have been a boon for improving the quality of textbooks and other instructional material. After all, there are only so many textbooks on the market addressing essentially the same educational standards — Common Core — so seriously reviewing them all should have been doable.
And, indeed, an organization called EdReports was established in 2014 to do precisely that: review textbooks. EdReports is a nonprofit, a major chunk of whose support comes from key promoters of the original Common Core. EdReports is dedicated to reviewing the alignment of textbooks with Common Core, yet its ratings are based only on “paper review,” not on any studies of the textbooks’ actual effectiveness.
Then, in 2017, the Council of Chief State School Officers (CCSSO), the originator of Common Core, established the High-Quality Instructional Materials and Professional Development (IMPD) Network dedicated to promoting materials rated well by EdReports as “High Quality” to member states. The key financing of this effort has been by the Bill and Melinda Gates Foundation, which funded its creation, and promoters that lend it credibility, such as the Fordham Institute.
All this would be fine if the Common Core standards were actually improving American education. But the standards are mediocre at best, and they have caused a clear deterioration of American student achievement in math and reading as visible on the 2017 and 2019 National Assessment of Educational Progress, or NAEP — the first declines in more than two decades. Here is what Education Department’s National Center for Education Statistics (NCES) had to say about the 2020 Long-Term Trend (LTT) NAEP, taken just weeks before the pandemic:
“The reading and mathematics scores of 13-year-old students fell between 2012 and 2020—the first time in the almost 50-year history of the National Assessment of Educational Progress (NAEP) long-term trend (LTT) assessment. … The mathematics scores for the lower-performing students (students at the 10th and 25th percentile) declined among students from both [9 & 13] age groups from the previous assessment in 2012. Scores also declined in mathematics for 13-year-olds at the 50th percentile. Scores for higher-performing students (at the 75th and 90th percentiles) did not change. … Mathematics scores at age 9 declined for females but did not change significantly for males since 2012.”
If it was not bad enough that EdReports rates textbooks on how well they are aligned with mediocre academic standards, it also verifies that textbooks do not stray from the Common Core standards’ scope and sequence (e.g., move standards to different grades) or spend a significant amount of time on additional standards. In other words, much of EdReports “quality” is embodied in checking whether textbooks closely follow Common Core rather than whether they present meaningful math or effectively teach it.
Then the Instructional Materials Network elevates those evaluations and promotes them as “High Quality” to states without any research to back it up.
You can see this with Singapore Math, which gained its fame because of its proven efficacy as demonstrated in Singapore’s high achievement on international mathematics tests. Yet, of the five textbooks based on Singapore Math, four of them receive EdReports’ lowest possible score, and one gets the second worst. And sure enough, the effectiveness of the texts in teaching math is not even checked by EdReports, which prefers a narrow and formalistic evaluation of effectiveness rather than actual empirical evidence. Similarly, Open Court Reading, which was responsible for improving the reading skills for millions of early grade students in California is deemed not worthy of evaluating its usability by EdReports as it doesn’t align with Common Core standards.
Perhaps in the future when parents are given a significant voice in their children’s choice of schools, such maverick textbooks will have a chance. Meanwhile, the Council of Chief State School Officers peddles untested textbooks hewing to mediocre standards on our teachers, giving them fake “High Quality” seals of approval and eliminating the chance to penetrate the wall of mediocrity.
•••
Ze’ev Wurman is a research fellow at the Independent Institute, chief software architect with MonolithIC 3D, and former senior policy adviser with the Office of Planning, Evaluation and Policy Development at the U.S. Department of Education.
The opinions in this commentary are those of the author. If you would like to submit a commentary, please review our guidelines and contact us.
To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.
Comments (9)
Comments Policy
We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.
Barry Garelick 4 months ago4 months ago
Excellent points. Many people think that a high rating from Ed Reports means the textbook is effective. Alignment with the Common Core standards is not a measure of effectiveness. In terms of Singapore Math, they got dinged because the standard algorithm for multidigit addition and subtraction was "assessed" in the textbook prior to 4th grade in which the standard appears. CC does not prohibit teaching the standard algorithms earlier than the grade in which they appear, … Read More
Excellent points. Many people think that a high rating from Ed Reports means the textbook is effective. Alignment with the Common Core standards is not a measure of effectiveness.
In terms of Singapore Math, they got dinged because the standard algorithm for multidigit addition and subtraction was “assessed” in the textbook prior to 4th grade in which the standard appears. CC does not prohibit teaching the standard algorithms earlier than the grade in which they appear, but the publishing guidelines prohibit the testing (i.e. “assessment”) of same in grades prior to when the std algorithm appears in CC. Note, it’s the publishing guidelines, not the CC standards themselves.
Dave G. 6 months ago6 months ago
Let the teacher decide!
Cathy Kessel 6 months ago6 months ago
The reference for “mediocre” is a report dated December 2017. Here are some examples of what it has to say about the Common Core standards for mathematics. Standard 1.OA.6 is criticized for its “long list of different ways of doing the operations.” In 2013, I examined a similar criticism from the same author, noting that the "different ways" appear as headings and objectives in Singapore Primary Mathematics 1A, Teacher’s Guide for the first half of grade … Read More
The reference for “mediocre” is a report dated December 2017. Here are some examples of what it has to say about the Common Core standards for mathematics.
Standard 1.OA.6 is criticized for its “long list of different ways of doing the operations.” In 2013, I examined a similar criticism from the same author, noting that the “different ways” appear as headings and objectives in Singapore Primary Mathematics 1A, Teacher’s Guide for the first half of grade 1. See comments-on-milgrams-review-of-final-draft-core-standards.
Other criticisms mention two poorly written problems which were not part of the CCSS.
The report also claims that “[4.NBT.4 Fluently add and subtract multi-digit whole numbers using the standard algorithm] is fully two to three years behind what is expected from students in high-achieving countries” and gives addition and subtraction examples that include 4- and 5-digit whole numbers from a translation of a grade 3 Russian textbook originally published in 1978 by Pcholko, Bantova, Moro, and Pyshkalo.
In contrast, the 2007 Singapore Syllabus says for grade 2 “Include: addition and subtraction of numbers up to 3 digits.” For grade 3, it says “Include: addition and subtraction of numbers up to 4 digits.” It doesn’t say anything further about whole-number addition and subtraction in grade 4, but it does include “addition and subtraction of decimals (up to 2 decimal places).” The 2012 Singapore Syllabus has similar expectations, see pp. 37, 42, 50 of the syllabus, which is here: https://www.moe.gov.sg/primary/curriculum/syllabus. (There is a new syllabus for the 2021 cohort but it doesn’t have entries beyond grade 2.)
The reference for “mediocre at best” is a report dated February 2010. The CCSS were released in June 2010: http://www.corestandards.org/about-the-standards/development-process/
Replies
Ze'ev Wurman 6 months ago6 months ago
Cathy Kessel is correct in that one of the references for the mediocrity of Common Core I quoted is from February of 2010, while the final version of Common Core was released only in June of 2010. So here is a report with similar finding on the final Common Core version, published in July 2010. https://pioneerinstitute.org/download/national-standards-still-dont-make-the-grade/ Kessel also finds an issue with the other paper I quoted, which argues Common Core "is fully two to three years … Read More
Cathy Kessel is correct in that one of the references for the mediocrity of Common Core I quoted is from February of 2010, while the final version of Common Core was released only in June of 2010. So here is a report with similar finding on the final Common Core version, published in July 2010.
https://pioneerinstitute.org/download/national-standards-still-dont-make-the-grade/
Kessel also finds an issue with the other paper I quoted, which argues Common Core “is fully two to three years behind what is expected from students in high-achieving countries” and demonstrates that when it comes to integer addition, it is “only” one year behind. There are other standards and expectations where Common Core is more behind but, eventually, all this is now moot, when we have actual empirical evidence of Common Core mediocrity.
On PISA international comparison, US declined between 2009 and 2018 by 9 points (from 487 to 478). On long-term NAEP our age 9 student declined 9 points (almost a grade level) between 2008 and 2019. And the C-SAIL report is the clincher, where the researchers embarked to find Common Core superiority and instead found that:
“Our study revealed that the adoption of the new college- and career-ready standards had significant negative effects in grade 4 reading. Specifically, our analysis suggests that had the “treatment” states stuck with their prior standards, their grade 4 NAEP reading scores would have been 2.3 to 3.8 points higher on the 0-500 NAEP scale during the seven years after the adoption of the new standards. We also found a significant negative seven-year impact on grade 8 mathematics. While most of our results were statistically non-significant, they tended to be in the negative direction.”
https://www.the74million.org/article/song-did-common-core-standards-work-new-study-finds-small-but-disturbing-negative-impacts-on-students-academic-achievement/
Cathy Kessel 6 months ago6 months ago
Re "demonstrates that when it comes to integer addition, it is “only” one year behind”: Does “Include: addition and subtraction of numbers up to 4 digits” mean “fluency with addition and subtraction of numbers up to 4 digits using a standard algorithm”? I don’t see anything in the Singapore teacher’s guides that says so. Also, note that grade 3 addition and subtraction expectations for Russia (then USSR) and for Singapore differ: one includes 5-digit numbers, … Read More
Re “demonstrates that when it comes to integer addition, it is “only” one year behind”: Does “Include: addition and subtraction of numbers up to 4 digits” mean “fluency with addition and subtraction of numbers up to 4 digits using a standard algorithm”? I don’t see anything in the Singapore teacher’s guides that says so. Also, note that grade 3 addition and subtraction expectations for Russia (then USSR) and for Singapore differ: one includes 5-digit numbers, the other does not. I believe this discrepancy may be due to an error in the 2017 “mediocrity” report’s account of Russian mathematics education history, and will link an account of details here: https://mathedck.wordpress.com/2013/08/04/comments-on-milgrams-review-of-final-draft-core-standards/.
On the page about the C-SAIL “report” (“Effects of states’ adoption of college- and career-ready standards on student achievement”), the reader is told that it was a working paper (that is, not peer reviewed). I write “was” because the link for the paper is dead. There is a more recent peer-reviewed article with a somewhat similar name (“Did States’ Adoption of More Rigorous Standards Lead to Improved Student Achievement? Evidence From a Comparative Interrupted Time Series Study of Standards-Based Reform”), somewhat different conclusions, and two of the same authors. Its abstract says:
“This study was designed to assess the effects of states’ adoption of more rigorous standards as part of the current wave of standards-based reform on student achievement using comparative interrupted time series analyses based on state-level NAEP data from 1990 to 2017. Results show that the effects of adopting more rigorous standards on students’ mathematics achievement were generally small and not significant. The effects on students’ reading achievement were also generally small, but negative and statistically significant for Grade 4. The study also revealed that the effects of states’ adoption of more rigorous standards varied across NAEP subscales and student subgroups.” https://journals.sagepub.com/doi/abs/10.3102/00028312211058460
Similarly, there were mixed results when, years ago, the Singapore books were used at four US sites. Here is an excerpt from the report (which was published in 2005 by AIR, the American Institutes for Research):
“The Montgomery County [Maryland] outcomes were positively correlated with the amount of professional training the staff received. Two Singapore pilot schools availed themselves of extensive professional development and outperformed the controls; two other pilot schools had low staff commitment coupled with low exposure to professional training and were actually outperformed by the controls. Professional training is important in helping teachers understand and explain the nonroutine, multistep problems in the Singapore textbooks. Teachers also need preparation to explain solutions to Singapore problems, which often require students to draw on previously taught mathematics topics, which the Singapore textbook, in contrast to U.S. textbooks, does not reteach.” https://www.air.org/resource/report/what-united-states-can-learn-singapores-world-class-mathematics-system-exploratory
After noting the variability in test site outcomes, the AIR report says: “our study shifted from merely assessing the results from the U.S. textbook pilots to developing a broad comparison of the coherence and quality of the Singapore and U.S. systems for delivering mathematics instruction.” For example, it notes “Singapore’s teachers must take a stringent examination before being accepted to education school, and while they are students, they are paid a teacher’s salary.”
Also, “Singapore’s professional development program offers sustained learning opportunities through a modularized approach that adjusts to teachers’ learning needs and is integrated into a continual learning process that includes experiential training experiences in nonschool settings. In contrast, professional training in U.S. school systems focuses on short-term workshops that fit into teachers’ released time. Evaluations suggest that these experiences are not likely to improve teachers’ performance in mathematics; the teachers themselves agree and do not seem to value this training very much.”
A recent PD example from Singapore: https://www.science.edu.sg/for-schools/teacher-professional-development/teacher-work-attachment.
A recent PD example from the US: https://edsource.org/2022/we-need-to-fix-professional-development-for-teachers/678632
As with the AIR study, the mixed results of the C-SAIL study may have prompted more focus on the US educational system, rather than a single component of the system. The Center on Standards, Alignment, Instruction, and Learning (C-SAIL) conference in 2020 was on “What’s next for standards-based reform? With the release of the 2019 National Assessment of Educational Progress results in math and reading, it became clear that standards-based reform has not moved the needle on student achievement. This may be due, in part, to how districts, schools, and teachers are making sense of and implementing college- and career-readiness standards.” https://www.c-sail.org/videos
Transforming expectations (such as standards) into instruction is no easy matter—as shown by three decades of standard-based reform. “We failed to prioritize the hard work of translating standards into rigorous curriculum materials, instructional strategies and teacher training,” says a recent essay (https://www.future-ed.org/unfinished-agenda-the-future-of-standards-based-school-reform/).
Ze'ev Wurman 6 months ago6 months ago
Cathy Kessel and I could continue this back-and-forth for a long time, yet a few facts are indisputable: The key ones are that the regular (Main) NAEP results started to level off and drop since the 2017 NAEP, after decades where they consistently grew. Similarly, the results for Long-Term (LTT) NAEP dropped for the first time between 2008 and 2019. PISA results, particularly in math, did the same for US students. US TIMSS achievement increased by 22 (!) points … Read More
Cathy Kessel and I could continue this back-and-forth for a long time, yet a few facts are indisputable:
The key ones are that the regular (Main) NAEP results started to level off and drop since the 2017 NAEP, after decades where they consistently grew.
Similarly, the results for Long-Term (LTT) NAEP dropped for the first time between 2008 and 2019.
PISA results, particularly in math, did the same for US students.
US TIMSS achievement increased by 22 (!) points between 1995 and 2015, yet *dropped* by by 4 points between 2015 and 2019.
All this occurred *before* the pandemic but after Common Core adoption.
Kessel is correct when she quotes “Transforming expectations (such as standards) into instruction is no easy matter – as shown by three decades of standard-based reform,” particularly in systems similar to ours where teachers are often encouraged not to follow recommendations based on robust research “invent” curricula and pedagogies on-the-fly behind their closed doors. Yet such consistent *national* (as opposed to local and regional) results across multiple national and international tests, of actual *decreases* in achievement since Common Core entered the mainstream of our classrooms, cannot be “explained” away as just “complexities of implementation,” as Kessel attempts.
There are many other issues with Kessel’s attempts to show that standards like Singapore’s or other high achieving countries are “similar” to Common Core, but this is inside baseball to most so I will not engage in it here.
Cathy Kessel 5 months ago5 months ago
People concerned about children’s education might appreciate clarification of claims about high-achieving countries. The 2017 “mediocrity” report Wurman cites says “[Common Core 4.NBT.4 Fluently add and subtract multi-digit whole numbers using the standard algorithm] is fully two to three years behind what is expected from students in high-achieving countries” (page 16, https://pioneerinstitute.org/pioneer-research/academic-standards-pioneer-research/mediocrity-2-0-massachusetts-rebrands-common-core-ela-and-math/). In contrast, Wurman is a co-author of a 2020 report that says “MA.3.NSO.2.1 Add and subtract multi-digit whole numbers including using a standard … Read More
People concerned about children’s education might appreciate clarification of claims about high-achieving countries.
The 2017 “mediocrity” report Wurman cites says “[Common Core 4.NBT.4 Fluently add and subtract multi-digit whole numbers using the standard algorithm] is fully two to three years behind what is expected from students in high-achieving countries” (page 16, https://pioneerinstitute.org/pioneer-research/academic-standards-pioneer-research/mediocrity-2-0-massachusetts-rebrands-common-core-ela-and-math/).
In contrast, Wurman is a co-author of a 2020 report that says “MA.3.NSO.2.1 Add and subtract multi-digit whole numbers including using a standard algorithm with procedural fluency” is “exemplary” (page 2, Better Than Common Core, https://www.independent.org/publications/article.asp?id=13176).
Perhaps a future Independent Institute report will explain this apparent contradiction.
NAEP results: As a commenter has pointed out, Minnesota did not adopt Common Core. Its grade 8 NAEP scores follow the national pattern.
https://nces.ed.gov/nationsreportcard/subject/publications/stt2019/pdf/2020013MN8.pdf
Despite the predictions of the “mediocrity” report (full title is Mediocrity 2.0: Massachusetts rebrands Common Core ELA & Math), grade 8 NAEP scores in Massachusetts were higher than Minnesota’s:
https://nces.ed.gov/nationsreportcard/subject/publications/stt2019/pdf/2020013MA8.pdf
One author has suggested that state investment in teachers is an important part of the explanation:
“Even more relevant for us is the experience of Massachusetts, which in the early 1990s had mathematics achievement that was fairly typical of the US, but which decided that that was not good enough, and embarked on a long-term effort to improve. The Massachusetts Education Reform Act of 1993 provided for increased funding to make opportunities to learn more equitable, substantial professional development, and higher mathematics requirements for teacher certification (thus impacting teacher preparation programs) [6]. It worked. Massachusetts now, considered as a separate country, has test scores significantly above the US average [20], [16], [11], [7]. Massachusetts also comes out at the top in the National Assessment of Educational Progress (NAEP), the US’s own instrument for assessing student achievement. For example, in the 2019 edition of NAEP, Massachusetts eighth-graders had the highest overall average of any state, and perhaps more significantly, 48% of them scored at the proficient or advanced level [12]. Only two other states had over 40% at these levels. The rest of the country should take some lessons.”
For references and the rest of the article, see https://www.ams.org/notices/202006/rnoti-p842.pdf.
Bruce William Smith 6 months ago6 months ago
Minnesota was wise to reject the Common Core mathematical standards while accepting those for English; One World Education Centre has long done the same. We use the Singapore mathematics textbooks in use in Singapore, rather than those being marketed as such in the United States, and our students consistently score in the 99th percentile on American math tests like the SAT and ACT, and on the exams for AP Calculus.
Linda Diamond 6 months ago6 months ago
This article is spot on. Ze'ev got it right. Not only is the Common Core dated, it's relationship to what we know from evidence is necessary to learn to read is tenuous at best. In fact, many excellent proven ELA programs and foundational skills curricula do not even bother submitting because although they have proven results, they know they will not fit the Common Core criteria. As Louisa Moats pointed out, the CCSS was weak … Read More
This article is spot on. Ze’ev got it right. Not only is the Common Core dated, it’s relationship to what we know from evidence is necessary to learn to read is tenuous at best. In fact, many excellent proven ELA programs and foundational skills curricula do not even bother submitting because although they have proven results, they know they will not fit the Common Core criteria. As Louisa Moats pointed out, the CCSS was weak from the start on word recognition and writing and spelling. Thank you Ze’ev!