Bigger dashboards may not steer districts in right direction

Lately I’ve been doing some musing about educational dashboards, the displays that school districts and others are developing to provide quick indicators of success or failure.

The iconography, of course, is derived from the dashboards in our automobiles, and we understand the self-correcting nature of those indicators. People who don’t heed the “tank empty” light frequently find themselves over at the side of the road calling the service truck. Zipping past the nice state patrol officer when the speedometer is pegged at 87 is likely to be expensive. Will this same kind of real-time information be useful in education? If so, what information should students, teachers, and parents get?

The Riverside Unified School District – a California leader in things digital – has created the five-indicator dashboard illustrated here. A student and, perhaps more importantly, his or her parents can know whether assignments are missing or classes skipped. The California High School Exit Exam and credits earned toward graduation show up in other gauges.

Vendors of what are called Learning Management Systems are developing dashboards like crazy, and the Michael and Susan Dell Foundation has built a powerful engine called ed-fi that sits behind the dashboard and connects it with school, district, state, and vendor systems. The State Board of Education recently raised the possibility of creating a dashboard for every school as part of the School Accountability Report Card, or SARC, that is posted on every school website.

Meanwhile, as Frank Catalano reports on Mind/Shift, the Council of State School Officers is hard at work creating a multi-state Shared Learning Collaborative, a big-bucket data warehouse that would be free to states and districts. They are also developing spigots for the buckets, apps that students and teachers could use to get data out.

The dashboards and the engines behind them are linked to an assumption that better data will yield better schooling. Educators frequently observe that a great deal of information is available in schools, but teachers and students seldom use it. There is stuff that teachers don’t know about their students that arguably would help them – things such as the pattern of mistakes students make in solving math problems – and there are things that students should know to help themselves, such as how to stay on the pathway to college.

Thus, it’s easy to make a strong case for dashboards as important devices to feed back data to students and schools. Making complex data actionable ought to be one of the benefits of computer technology for learning.

That said, the current crop of dashboards seem pretty limited, and overreliance on dashboard data can create horrible goal substitution in which turning all the dashboard lights green equates with learning or a good education. (There are also bunches of privacy issues that I won’t deal with here.)

A better, less limited dashboard would be personalized. Data elements should work like Widgets or elements on a computer home page. A student who has passed the high school exit exam and is aiming toward Cal or Harvey Mudd doesn’t need the CAHSEE icon staring him or her in the face. Let students get the gauges they need.

We do that with car dashboards. Even my seven-year-old auto lets me change the displays, and as Claremont Graduate University alum Mark Maine, who knows a thing or two about cars, writes:

Racecars can both send and receive data … lots of data. For the driver, the instruments, including things like shift lights, tell him/her about their performance with detailed info on sector and lap times. They have this type of technology at the go-kart level all the way up to Formula 1. In addition, teams can monitor the car’s performance including tire pressure changes and share that info with the driver so he or she is in constant two-way communication, like real time OnStar. Finally, teams collect and analyze tons of data from different parts of the car to improve the car’s setup for a particular track and driver. They also pay a great deal of attention to what the driver has to say about the car’s performance and track behavior.

Consider the elements of a racecar dashboard and the data system behind it. First, a great deal of data is analyzed in real time. Both the driver and the crew analyze and react to data. The crew customizes the car for each race and each driver. The crew listens carefully to what the driver says about his or her experience. The data are useful because they are current, all parties know what to do with them, and the parts of the system communicate, both electronically and in person.

We have only begun to develop similar adaptive capacity with educational data. Thus, the first dashboard danger is that the information will be irrelevant or static. If a dashboard doesn’t substantially improve information flows from those available in old-fashioned report cards, it isn’t worth the effort.

However, the greater danger in dashboards is that we might come to confuse the variables they report with the goals of education. The goal of driving is not to keep from running out of gas, or to prevent the car from boiling over. Those are only conditions for keeping the car in motion. No one would confuse those indicators with a beautiful trip or a good driver.

When I was driving home from the office the other day I pulled up behind a car whose license plate holder said “EXCUSE me for driving the speed limit.” Actually, the driver was loping along at about 4 mph under. I am sure the driver was pleased with all the green light indicators on his dashboard as he, oblivious to his surroundings, held up a line of traffic all the way through town.

It is highly likely that uncritical policy viewers and school administrators will engage in the same kind of goal substitution. Making all the indicators turn green is not the same as learning. But the better we get at creating and customizing dashboards, the more proximate the two will be.

Charles Taylor Kerchner is Research Professor in the School of Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy, and teachers unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District. The results of that research can be found in The Transformation of Great American School Districts and in Learning from L.A.: Institutional Change in American Public Education, published by Harvard Education Press. This column first appeared on his blog, Mindworkers.com. Readers with examples of real-time indicator systems are requested to contact him there.

For previous commentaries that Charles Taylor Kerchner wrote for Thoughts on Public Education (TOP-ed.org) and for EdSource, go here.

To get more reports like this one, click here to sign up for EdSource’s no-cost daily email on latest developments in education.