We Need a More Robust Learning Sciences Research Community

My latest Chronicle column is on how inherently difficult it is to evaluate learning science claims, particularly when they get boiled down to marketing claims about product efficacy, and how deep academic distrust of vendors makes this already incredibly difficult challenge nearly impossible.

Here’s where I stand on vendor participation in ed tech and learning science research:

On the one hand, vendors have access to data and resources that, for a variety of reasons, are difficult or even impossible for universities to access. They sometimes have millions of students using their products and relatively few internal barriers to doing certain kinds of large-scale effectiveness research (although to a certain degree, territoriality, bureaucracy, and poorly designed data architectures are universal problems). On the other hand, vendors should not under any circumstances be allowed to define the research agenda and arbitrate the validity of learning sciences claims for all of education. Theoretically, there is a simple solution to this. Higher ed knows how to do science. It knows how to review scientific claims. Vendor research should be put through the peer review process. They should publish enough information that their results can be evaluated and, when feasible, duplicated. Their reputations should be based in part on who is doing (or, at least, properly using) good scientific research. After all, most of the researchers at these companies are PhDs who are trained in the academic research process. There is no reason in the world why these companies can’t contribute to real, rigorous, peer-reviewed dialog and progress in the field.

Unfortunately, the current state of the learning sciences research community is weak and fragmented. There is incredibly good work going on in pockets here and there, but overall, it’s a mess. This is bad for many reasons. It both lets vendors get away with junk science claims while failing to reward good behavior. Worse, it effectively abdicates the central role that academia should be playing in driving the research agenda.