I wish that discussions of education data were as interesting as my conversations on baseball statistics. Contrary to my reputation as an Ed Dork, I do not wake up every morning, grab my phone and check out EdWeek, EdSource and Eduwonk for the latest Ed news. I wake up, grab my phone and check out two pretty amazing baseball blogs: crashburnalley.com and fangraphs.com.
The first was created by amateur statisticians and hardcore fans of the Philadelphia Phillies. The second is the online bible for baseball statistics junkies nationally. Both sites view the game through the lens of numbers. The movement of those numbers up and down reflects the performance of ballplayers. The beauty of these sites is that they’ve taken the numbers that were once the province of baseball lifers and general managers and democratized them. This has resulted in the proliferation of metrics such as OPS (On-base Plus Slugging), WAR (Wins Above Replacement), BABIP (Batting Average on Balls in Play), and countless others. Over the past decade, as these metrics have proliferated, it’s hard to know which ones have been created by professionals and which ones by amateurs. In fact, the stats revolution has moved beyond baseball, taken over basketball and started to change football.
So, why is education still so old school?
Take the debate over testing. At one extreme, you have people bashing standardized testing of any kind. On the other, you have people supporting testing, but typically focusing on just two data points (English and math). The first perspective just seems silly. It’s like saying that we shouldn’t measure batting average and instead just look at hitters to see whether they’re good. The second seems insanely limiting – as in, we should only measure batting average and runs batted in (which is basically what baseball did for most of its history).
It would be much more productive to have a 21st century conversation about how all data are good. Instead of just two data points, we should use as much data as necessary to paint a complete picture of student performance. This means accepting student data beyond academic measures, like social-emotional learning, and investing in new ways of assessment such as portfolios and exhibitions. In combination, all of these data can present a far more interesting and realistic picture of a student’s strengths and needs than just grades and tests. Indeed, the proliferation of multiple data points in baseball has highlighted the potential of players who would have been overlooked or completely ignored in earlier times.
The other lesson that baseball can teach education is that the people who have always been in charge of the numbers are not in charge anymore. Anyone can now become an amateur statistician. Some reports about district accountability, continuous improvement and the state’s role in determining school quality could have been written 20 years ago. They presume that the state is still in charge of all education data and their presentation. That may still have a little truth now but it won’t be true much longer. Statistics, particularly those paid for by taxpayers, are in the public domain. And smart people are going to come up with their own presentations of those data for public consumption, especially if they can make a buck off it.
This means that discussions on coming up with the latest policy innovations like “dashboard presentations” of districts and school performance vs. using a single indicator like the Academic Performance Index (API) will soon become moot. The state may come up with dashboards. Private companies and education stats junkies may come up with dashboards. But if they are too difficult for the average consumer of this information to use, stakeholders ranging from parents to homeowners will find an alternative more similar to the API that bundles and weights all of these indicators into a composite score. Again, when you look at baseball, it’s only really a small percentage of the baseball fans who have the inclination to delve into Fangraphs and the glories of Win Probability Added (WPA). To resolve this, the stats geeks created a composite metric called Wins Above Replacement (WAR) that is increasingly used by the average fan. Now, I might not like the API. I might believe that it’s the wrong way to look at the complexity of schools. But I also know that I live in a world that ranks everything with numbers or grades, from my restaurant, to my Uber driver, to my graduate school.
This leads me to my last point. We live in a world, very different from the old world, where data are also inevitable. How many times have you checked Facebook or LinkedIn, shopped online, texted your friend, used the word “weather” in a search engine today? You may not have counted, but someone else has. Last month, I talked to a friend at one of the largest consulting companies in the world. It is now using an algorithm that looks at keywords in résumés and cover letters and screens out candidates based on words and phrases used by previous unsuccessful candidates. A similar trend is happening in sports, where every movement of an athlete is captured as a data point. One day everything we do in the digital world will paint a picture of our competency or lack thereof. Regardless of whether you support or oppose this trend, it is becoming our students’ reality.
As policymakers navigate this new world they will have to come to grips with the limitations of their power. They will have to factor in the impact of “bottom-up” solutions in areas they previously controlled, such as ratings of school performance (see greatschools.org and reportcards.edtrustwest.org) and measurements of college readiness. Instead of trying to stamp out these efforts, they will have to think very differently about issues of accountability, transparency and privacy. Otherwise, they will become just as irrelevant as old-school baseball traditionalists.
To date, the proliferation, democratization and inevitability of data in sports has been immensely beneficial. If we start with an open mind and work to guide instead of control the use of data, it can have the same positive impact in education.
•••
Arun Ramanathan is the CEO of Pivot Learning Partners.
EdSource welcomes commentaries representing diverse points of view. The opinions expressed in this commentary represent those of the author. If you would like to submit a commentary, please review our guidelines and contact us.
To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.
Comments (7)
Comments Policy
We welcome your comments. All comments are moderated for civility, relevance and other considerations. Click here for EdSource's Comments Policy.
Jeff Camp 7 years ago7 years ago
I wish that education statistics were available as promptly and reliably as baseball stats. It makes no sense to me in that education data is still collected and assembled as if it were being hand-delivered, read aloud for tabluation on papyrus and distributed by pigeon. The best reports that the public can get aren't actually disseminated as data -- they are dumped as PDFs, which are essentially pictures. Each school district can use its own … Read More
I wish that education statistics were available as promptly and reliably as baseball stats. It makes no sense to me in that education data is still collected and assembled as if it were being hand-delivered, read aloud for tabluation on papyrus and distributed by pigeon. The best reports that the public can get aren’t actually disseminated as data — they are dumped as PDFs, which are essentially pictures. Each school district can use its own format. Districts can “make them available to the public” in any way the spirit moves them, which makes comparisons difficult, expensive, slow and unreliable. EdTrustWest does great work slogging through these local documents in an effort to turn them back into data, but they can’t collect everything. ChildrenNow does a bunch of this work, too. NEA takes on the most ambitious task, trying to characterize the high-level numbers across the whole nation. NCES seems to rely deeply on NEA’s efforts. How is it not a basic function of government to have a consistent mechanism to promptly collate the facts? Education is far, far behind baseball in this arena.
Don 7 years ago7 years ago
“…the proliferation of multiple data points in baseball has highlighted the potential of players who would have been overlooked or completely ignored in earlier times.”
The accountability purposes of standardized testing have little to do with identifying diamonds in the rough. Moreover, the type of testing employed is unlikely to reveal such diamonds. Mr. Ramanathan’s analogy between baseball stats and testing stats is not clear to me.
navigio 7 years ago7 years ago
One things for sure, the term 'moneyball' would apply well to education 'reform'. :-) Although it's possible to argue baseball players are better than they used to be (at least in terms of strength and size), it's unlikely that is due to fans becoming interested in stats. It's also not clear that this has made the game of baseball itself any 'better'. What stats have done is change how players are used in the form of … Read More
One things for sure, the term ‘moneyball’ would apply well to education ‘reform’. 🙂
Although it’s possible to argue baseball players are better than they used to be (at least in terms of strength and size), it’s unlikely that is due to fans becoming interested in stats.
It’s also not clear that this has made the game of baseball itself any ‘better’.
What stats have done is change how players are used in the form of increased specialization (and an associated increased use of the DL–are you offering a DL for teachers who suffer the stress of being exploited by the system?).
But in the end, the result is still the same: a single World Series winner and usually a different one. The effect is between teams and in a transitory way. In the meantime, many records remain that existed prior to ocd stat-keeping.
May also be worth noting the proliferation of stats that try to ‘adjust’ for external factors.
Maybe it’s time to implement a law that forbids anyone who can’t make the majors from playing baseball at any level.
Btw, I also don’t read edu sites first thing in the morning. I wake up in the middle of the night so I don’t have to wait that long..
Steve Rees 7 years ago7 years ago
Arun poses a powerful question. How can the discussion of baseball stats be so rich, and the discussion of ed stats be so poor? My hunch is that education is so dominated by practitioners that analysts have no seats at the table. Medicine has been enjoying a dynamic tension in the debate between those who favor evidence-based practices, and those who favor reliance on clinicians' judgment. Now diagnostic decision-support software is being used by those … Read More
Arun poses a powerful question. How can the discussion of baseball stats be so rich, and the discussion of ed stats be so poor? My hunch is that education is so dominated by practitioners that analysts have no seats at the table. Medicine has been enjoying a dynamic tension in the debate between those who favor evidence-based practices, and those who favor reliance on clinicians’ judgment. Now diagnostic decision-support software is being used by those who want help making diagnostic judgments, while those who are too proud to turn for suggestions to a “computer” shun it entirely.
The field of psychology was torn by debate when one of its young leaders, Paul E. Meehl, published a paper in 1954 titled “Clinical vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence.” He argued that statistical analysts could successfully diagnose schizophrenia at a higher rate than clinicians who examined the patients in person. The controversy sparked by Meehl’s paper hasn’t settled down in the past sixty years.
What will it take for the debates in the castle of K-12 to mature to the point that enables us to transcend the blunt, binary debate over testing (“are you pro-testing or anti-testing?”). Arun’s challenging question deserves an answer. Perhaps education leaders and educators of educators would benefit from the cross-pollination of people and knowledge that benefits other professions — sciences, sports and business. My hunch is that the insularity of the profession of education management is one of many barriers to a richer debate. The good news — it is a barrier that is removable.
Replies
Floyd Thursby 7 years ago7 years ago
Stats are very important in education. Analyzing them will really help us understand how to help poor kids better. That's why Amy Chua's book with her husband (not the first one, the Triple Package) illuminated how home culture can influence educational achievement and suggests ways we can all adjust our family cultures to improve our kids' test scores by making it more similar to the key groups (Chinese, Nigerian, Persian, Lebanese, Korean, Indian, … Read More
Stats are very important in education. Analyzing them will really help us understand how to help poor kids better. That’s why Amy Chua’s book with her husband (not the first one, the Triple Package) illuminated how home culture can influence educational achievement and suggests ways we can all adjust our family cultures to improve our kids’ test scores by making it more similar to the key groups (Chinese, Nigerian, Persian, Lebanese, Korean, Indian, Kenyan, many others).
In general, the stats show richer kids do better, but that is hard to solve. The exceptions are the Triple Package stats and some charter school methodologies like KIPP and Harlem Zone. I’m all for making it easier to fire bad teachers and having some merit pay, a la Vergara, but I think a statistical analysis will show that will have less than a 5% impact. There is a difference between a great and mediocre teacher, but only a few will be fired, or deserve to be, and there will still be randomization, so firing a few teachers won’t reverse the achievement gap substantially.
I believe instead of consultants or professional development, hiring one-on-one tutors for the poorest kids would make a difference. Stats are key because I’ve been wrong and many have been wrong, so stats are a truth teller.
I thought cash bonuses would make poor kids try harder, but they don’t. Metrics can’t lie. A Money Ball approach to education is a good idea. Affirmative Action comes too late and then kids can’t compete. I believe we should have affirmative action from a young age, more taxes going to tutors for the poor, less going to imprisoning the poor or building more nuclear weapons or on consultants. Part of the problem is many people have a vested interest and approach it with bias, be it pro union, anti union, or a paranoia that people are blaming the poor rather than trying to help them, or a desire of the rich to keep the poor down, or a fear of racism, founded or unfounded. I want to see a world in which every child has an equal chance at success, rich or poor, black, white, Latino, Asian, gay, straight. We don’t have that now. We need to study factors. We know that Amy Chua showed the cultures who thrive have a seemingly extreme or irrational need to prove one’s self statistically, to prove they are good, a combination of a hope of superiority with a chip on one’s shoulder or insecurity.
You notice some rich kids get lazy, they’re good enough. You need a constant feeling of insecurity driving you to work hard, study hard, and parents being determined not to divorce, not to let their kids waste their summers, etc. Every child can succeed if his or her parents give it their best effort, and they have teachers who are competent and care.
navigio 7 years ago7 years ago
Give it up. Charter schools have the ability to fire whomever they want, yet their achievement gap persists. Time to find solutions that help kids, not adults.
Don 7 years ago7 years ago
In theory, yes, Navigio, but that is also true for unionized teachers, according to the Vergara defendants. In any case, I agree that on balance it is easier for a non-union charter school to dismiss a teacher, but finding new ones is difficult and makes dismissal less of an option in practice if not in theory. I complained vociferously about a grossly incompetent teacher at my son's charter school and they ignored my … Read More
In theory, yes, Navigio, but that is also true for unionized teachers, according to the Vergara defendants. In any case, I agree that on balance it is easier for a non-union charter school to dismiss a teacher, but finding new ones is difficult and makes dismissal less of an option in practice if not in theory. I complained vociferously about a grossly incompetent teacher at my son’s charter school and they ignored my concerns until that teacher just up and left one day. Unfortunately, we opted to leave before he voluntarily spared the students of his incompetence. Silly me for thinking I wouldn’t have this problem at a charter.