Whatever else you think of the election, it has been the mother of all teachable moments for many of us. It has raised questions about what we thought we knew about our democracy, our neighbors, our media…and apparently learning analytics. The shock of the polls being “wrong” has raised a lot of questions about how much we can really trust data analytics. Audrey Watters has written the most fleshed out critique that I’ve seen so far. But Dave Cormier tweeted about it as well. And I have had several private conversations along these lines. They all raise the question of whether we put too much faith in numerical analysis in general and complex learning analytics in particular. That is an excellent question. But in doing so, some of these arguments position analytics in opposition to narratives. That part is not right. Analytics are narratives. They are stories that we tell, or that machines tell, in order to make meaning out of data points. The problem is that most of us aren’t especially literate in this kind of narrative and don’t know how to critique it well.
This is going to be a wide-ranging post that goes a little lit crit at times and dives into an eclectic collection of topics from election polling to the history of medicine. But even the most pragmatic, b-school-minded entrepreneur or VC may find some value here. Because what I’m ultimately talking about is a fundamental limiter on the future growth of the ed tech industry. The value of learning analytics, and therefore the market for them, will be limited by the data and statistical literacy of those who adopt it. The companies that are focused on developing fancier algorithms are solving the wrong problem—at least for now. These tools will have limited adoption until they are put into the hands of educators who understand their uses and limitations. And we have a long way to go in that department.