My comments in today’s NPR article about Knewton are getting some attention on Twitter. One comment in particular, actually. The one where I accused Knewton CEO Jose Ferriera of selling snake oil. To understand the basis of the comment, you have only to look at the quote from Jose earlier in that selfsame article:
We think of it like a robot tutor in the sky that can semi-read your mind and figure out what your strengths and weaknesses are, down to the percentile.
Can it read what I’m thinking now?
And then there’s this:
We can take the combined data power of millions of students — all the people who are just like you — [who] had to learn a particular concept before, that you have to learn today — to find the best pieces of content, proven most effective for people just like you, and give that to you every single time.
Really? Every single time?
Objectively speaking, I don’t know much about Knewton’s platform or the value that it adds. The only efficacy studies I have seen are for Pearson products, and those studies do not differentiate between Knewton features and Pearson-native features in terms of impact. If you dig deep enough on their site, you can find some information on the techniques that they use. For example, this overview paper, while not perfect, is a credible attempt to explain the techniques that the platform employs and the benefits to the users in layperson’s terms. If Knewton were doing more of this, I wouldn’t be as critical. There are still questions about the value of the product, but they are on the same level with many other products in the adaptive learning category. A company like Knewton should be working hard to demystify what their product does and providing hard, reproducible evidence that they add value.
But much of what Jose says, at least to the media, is the opposite. No responsible educator or parent should adopt a product—even if it is free—from a company whose CEO describes it as a “robot tutor in the sky that can semi-read your mind” and give you content “proven most effective for people like you every single time.” I’m sorry, but this sort of quasi-mystical garbage debases the very notion of education and harms Knewton’s brand in the process.
If you want to sell me a product that helps students to learn, then don’t insult my intelligence. Explain what the damned thing does in clear, concrete, and straightforward language, with real-world examples when possible. I may not be a data scientist, but I’m not an idiot either. If you can translate the technical mumbo jumbo into sensible teaching insights for me, then you just might make a sale. But most people wouldn’t buy a used car from somebody who describes their product using language as hyperbolic as “robot tutors in the sky.” The same principle should hold for educational technology.
I do use a Pearson product for my students with the Knewton adaptive learning. As far as I can tell, the Knewton part looks at the math problems the students get wrong and then spews out a set of related practice exercises. Students get a plan for studying with so many problems that it is not really an efficient study plan. I just tell my students to pick and work the ones they have the most trouble with.
It’s interesting to note that the Knewton website is always looking for programmers but never any content writers. Huh? I think they have generic programs which hook up to giant databases which map relationships between problem types. At least that’s what it looks like in the math product from what I have seen.