College & Careers > College Ready

Bigger dashboards may not steer districts in right direction



Charles Taylor Kerchner

Lately I’ve been doing some musing about educational dashboards, the displays that school districts and others are developing to provide quick indicators of success or failure.

The iconography, of course, is derived from the dashboards in our automobiles, and we understand the self-correcting nature of those indicators. People who don’t heed the “tank empty” light frequently find themselves over at the side of the road calling the service truck. Zipping past the nice state patrol officer when the speedometer is pegged at 87 is likely to be expensive. Will this same kind of real-time information be useful in education? If so, what information should students, teachers, and parents get?

Riverside Unified’s dashboard lets students know assignments that are missing and credits needed for graduation. Click to enlarge.

The Riverside Unified School District – a California leader in things digital – has created the five-indicator dashboard illustrated here. A student and, perhaps more importantly, his or her parents can know whether assignments are missing or classes skipped. The California High School Exit Exam and credits earned toward graduation show up in other gauges.

Vendors of what are called Learning Management Systems are developing dashboards like crazy, and the Michael and Susan Dell Foundation has built a powerful engine called ed-fi that sits behind the dashboard and connects it with school, district, state, and vendor systems. The State Board of Education recently raised the possibility of creating a dashboard for every school as part of the School Accountability Report Card, or SARC, that is posted on every school website.

Meanwhile, as Frank Catalano reports on Mind/Shift, the Council of State School Officers is hard at work creating a multi-state Shared Learning Collaborative, a big-bucket data warehouse that would be free to states and districts. They are also developing spigots for the buckets, apps that students and teachers could use to get data out.

The dashboards and the engines behind them are linked to an assumption that better data will yield better schooling. Educators frequently observe that a great deal of information is available in schools, but teachers and students seldom use it. There is stuff that teachers don’t know about their students that arguably would help them – things such as the pattern of mistakes students make in solving math problems – and there are things that students should know to help themselves, such as how to stay on the pathway to college.

Thus, it’s easy to make a strong case for dashboards as important devices to feed back data to students and schools. Making complex data actionable ought to be one of the benefits of computer technology for learning.

That said, the current crop of dashboards seem pretty limited, and overreliance on dashboard data can create horrible goal substitution in which turning all the dashboard lights green equates with learning or a good education. (There are also bunches of privacy issues that I won’t deal with here.)

A better, less limited dashboard would be personalized. Data elements should work like Widgets or elements on a computer home page. A student who has passed the high school exit exam and is aiming toward Cal or Harvey Mudd doesn’t need the CAHSEE icon staring him or her in the face. Let students get the gauges they need.

We do that with car dashboards. Even my seven-year-old auto lets me change the displays, and as Claremont Graduate University alum Mark Maine, who knows a thing or two about cars, writes:

Racecars can both send and receive data … lots of data. For the driver, the instruments, including things like shift lights, tell him/her about their performance with detailed info on sector and lap times. They have this type of technology at the go-kart level all the way up to Formula 1. In addition, teams can monitor the car’s performance including tire pressure changes and share that info with the driver so he or she is in constant two-way communication, like real time OnStar. Finally, teams collect and analyze tons of data from different parts of the car to improve the car’s setup for a particular track and driver. They also pay a great deal of attention to what the driver has to say about the car’s performance and track behavior.

Consider the elements of a racecar dashboard and the data system behind it. First, a great deal of data is analyzed in real time. Both the driver and the crew analyze and react to data. The crew customizes the car for each race and each driver. The crew listens carefully to what the driver says about his or her experience. The data are useful because they are current, all parties know what to do with them, and the parts of the system communicate, both electronically and in person.

We have only begun to develop similar adaptive capacity with educational data. Thus, the first dashboard danger is that the information will be irrelevant or static. If a dashboard doesn’t substantially improve information flows from those available in old-fashioned report cards, it isn’t worth the effort.

However, the greater danger in dashboards is that we might come to confuse the variables they report with the goals of education. The goal of driving is not to keep from running out of gas, or to prevent the car from boiling over. Those are only conditions for keeping the car in motion. No one would confuse those indicators with a beautiful trip or a good driver.

When I was driving home from the office the other day I pulled up behind a car whose license plate holder said “EXCUSE me for driving the speed limit.” Actually, the driver was loping along at about 4 mph under. I am sure the driver was pleased with all the green light indicators on his dashboard as he, oblivious to his surroundings, held up a line of traffic all the way through town.

It is highly likely that uncritical policy viewers and school administrators will engage in the same kind of goal substitution. Making all the indicators turn green is not the same as learning. But the better we get at creating and customizing dashboards, the more proximate the two will be.

Charles Taylor Kerchner is Research Professor in the School of Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy, and teachers unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District. The results of that research can be found in The Transformation of Great American School Districts and in Learning from L.A.: Institutional Change in American Public Education, published by Harvard Education Press. This column first appeared on his blog, Mindworkers.com. Readers with examples of real-time indicator systems are requested to contact him there.

For previous commentaries that Charles Taylor Kerchner wrote for Thoughts on Public Education (TOP-ed.org) and for EdSource, go here.


 

Filed under: College Ready, Commentary, Data, Reforms

Comments

EdSource encourages a robust debate on education issues and welcomes comments from our readers. The level of thoughtfulness of our community of readers is rare among online news sites. To preserve a civil dialogue, writers should avoid personal, gratuitous attacks and invective. Comments should be relevant to the subject of the article responded to. EdSource retains the right not to publish inappropriate and non-germaine comments. EdSource encourages commenters to use their real names. Commenters who do decide to use a pseudonym should use it consistently.

Leave a Comment

Your email address will not be published. Required fields are marked *

 characters available

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

4 Responses to “Bigger dashboards may not steer districts in right direction”

  1. Charles Taylor Kerchner said

    on July 29, 2012 at 8:23 pm

    Richard and Chris,

    Thanks for taking the time to write.

    I’m not sure where you’ve been, or not been, but surprise at the use of electronic data feedback suggests an acute unawareness of the policy landscape in public education. LOTS of vendors, think tanks, entrepreneurs, and real school districts, and individual teachers are spending a substantial amounts of money, time and effort in creating these systems. Which is why I wrote about it, yes, with a straight face. And FYI, Richard, I am not being paid.

    Chris’ point cuts closer to the bone. It’s very important if we are going to use indicators that we pick the right ones and that the data are actually useful to students. It’s possible that indicator systems may be very helpful, and possible that they will be a waste of money and time. Surely, a dashboard will not replace a art teacher, but a dashboard may help a student and attentive adults realize that he or she needs just one more course to qualify for admission to the state university system.

  2. navigio said

    on July 26, 2012 at 3:33 pm

    I’m surprised by these comments.

    Conceptually speaking, the goal of making this kind of data more readily accessible is to make it easier for parents to be involved in a child’s education. Especially for those who cannot meet with teachers on an ongoing basis. The current ‘alternative’ is simply to hope the parent responds to messages sent home and is able to address the issues with the student.

    Clearly this would allow a student to much more easily stay on track and on goal. And maybe even be more likely to graduate, or successfully complete a-g, or similar. Not sure why that should be considered a waste of resources.

    Although the data in this particular dashboard is more directed toward graduation and course completion requirements, I expect there are others that delve deeper, the way a site council, principal or teachers currently might do anyway, albeit at much greater time expense. Whether that level of accountability is appropriate probably boils down to whether you feel accountability is appropriate in general. But thats a separate distinction.

    I also think those who think the analogy misses the point actually misunderstand some of the wording. In looking at the display, it seems obvious that the goal is not only to provide quick data (as in turnaround time between performance and response) but quick as in easily accessible. In other words, its much ‘quicker’ to have a gas gauge on your car’s dashboard than it is to open the cap, jiggle the car, and listen for the frequency of the waves splashing around. Its also probably much ‘quicker’ to have a new, modern gas level detection system, than an old float that is inaccurate and slow to react.

    Anyway, I think its going to be the goal of most school districts to focus on technology to try to cover the roles of humans anywhere they can given the fact that california chooses not to want to fund much of these positions anymore. Im not necessarily one for blind adoption of technology-based methods (even coming from the tech world), but its clear we could be making much better use of technology than we currently are, especially in transparency and data accessibility.

  3. Chris Bertelli said

    on July 26, 2012 at 2:24 pm

    You hit on the failure of the analogy (as well as the concept, in some cases) of these displays to car dashboards. If your gas tank reads empty, you know how to get it back to full. If you are speeding, you know how to reduce the speed. The problem with tools designed to “provide quick indicators of success or failure” is that they don’t tell you how to move from failure to success. Presenting data and using data aren’t the same thing. Unfortunately some are slow to realize that.

  4. Richard Moore said

    on July 26, 2012 at 11:29 am

    Wow. This is amazing stuff. Do schools actually indulge in this kind of nonsense? With no money for librarians, nurses, art teachers, music programs? Geeks with computers sit in rooms and design gizmos to tell us the temperature of the child who is being neglected? Really? And this guy can keep a straight face writing about it? Of course. He is getting paid to believe this . . . By all means, let’s raise taxes to spend more money on this nonsense.

Template last modified: