e-Literate

Present is Prologue

Tag: OLI

  • A Taxonomy of Adaptive Analytics Strategies

    I almost never quote a blog post in its entirety, but this one from Dan Meyer is so good that I just can’t bear to cut a single word:

    Stephanie Simon, reporting for Reuters on inBloom and SXSWedu:

    Does Johnny have trouble converting decimals to fractions? The database will have recorded that – and may have recorded as well that he finds textbooks boring, adores animation and plays baseball after school. Personalized learning software can use that data to serve up a tailor-made math lesson, perhaps an animated game that uses baseball statistics to teach decimals.

    Three observations:

    One, it shouldn’t cost $100 million to figure out that Johnny thinks textbooks are boring.

    Two, nowhere in this scenario do we find out why Johnny struggles to convert decimals to fractions. A qualified teacher could resolve that issue in a few minutes with a conversation, a few exercises, and a follow-up assessment. The computer, meanwhile, has a red x where the row labeled “Johnny” intersects the column labeled “Converting Decimals to Fractions.” It struggles to capture conceptual nuance.

    Three, “adores” protests a little too much. “Adores” represents the hopes and dreams of the educational technology industry. The purveyors of math educational technology understand that Johnny hates their lecture videos, selected response questions, and behaviorist video games. They hope they can sprinkle some metadata across those experiences — ie. Johnny likes baseball; Johnny adores animation — and transform them.

    But our efforts at personalization in math education have led all of our students to the same buffet line. Every station features the same horrible gruel but at its final station you can select your preferred seasoning for that gruel. Paprika, cumin, whatever, it’s yours. It may be the same gruel for Johnny afterwards, but Johnnyadores paprika.

    Dan captures most of what I was trying to get at with my rant on the big data hype, but much more clearly and succinctly. Points two and three are the most salient here. First of all, the sort of surface-level analysis we can get from applying machine learning techniques to the current data we have from digital education system is insufficient to do some of the most important diagnostic work that real human teachers do. Think about the math classes is which you had to show your work on your homework. Why was that important? Because the teacher needs to see not only what you got wrong but why you got it wrong. Teachers generally don’t just say, “You got three out of five problems involving converting decimals to fractions wrong. Go study some more.” They sit down and work through the problems with the student to find the source of the errors. It’s really hard to get computers to do this well, even with highly procedural domains like math. (Forget about, say, literary analysis.) So in the vast majority of cases, we don’t even try to design systems where students show their work. And without the step-by-step data, no fancy algorithm is going to teach Johnny.

    Second, if the problem is that your content isn’t what the student needs, no fancy algorithm is going to fix that either. Videos are a prime example. I know of one textbook publisher whose teacher customers report that students won’t watch the publishers’ videos, but they can and do find videos on the same topic on YouTube and share them with each other. Think about that. Video-based pedagogical support is valuable enough to the students that they will expend energy searching for videos and sharing them. But they reject the expensive, carefully crafted videos from the publisher that are served up to them on a silver platter. It’s not that the publisher-supplied videos are necessarily “bad” in the sense that they have poor production qualities or are unclear or factually inaccurate. But the students have a particular use in mind for the videos. Maybe they’re struggling with a particular homework problem and just need a quick walk-through of a technique so that they can see the step that they are missing, for example. If the video doesn’t fit their needs—both utilitarian and aesthetic—then it won’t get used. Serving it up adaptively isn’t going to help that problem.

    That said, it’s worth taking a little time to break down the different types of adaptive learning analytics into a couple of categories and see just what we should and should not reasonably hope to gain from them.

    (more…)

  • Open Secret: Pittsburgh’s Ed Tech Revolution

    Generally when we talk about goals for educational technology, we talk about one of two things: improving access or improving effectiveness. Rarely do we get an opportunity to talk credibly about an innovation that can move both of those needles at the same time. And yet, I saw just such innovation in Pittsburgh this week at the LearnLab Corporate Partners Meeting. LearnLab, jointly run by Carnegie Mellon University and the University of Pittsburgh, is not widely known in ed tech circles. However, it’s close cousin OLI is. For example, an independent study by ITHAKA S+R made a splash when it showed OLI could cut instructional time in half and still achieve the same level of effectiveness. Two groups of students across six different public universities took the same introductory statistics class. The first group took a traditional version of the class, with an average of 3 hours per week spent in the classroom. The second group took a hybrid version, where they spent one hour a week in the classroom and some time studying independently with the OLI cognitive tutoring software. So that second group spent one-third of the time in the classroom, which amounts to a very significant cost savings and an opportunity to teach more students for less money. Students themselves spent more time on homework in the second group than the first, but even so, they spent 25% less total time on the class than the ones in the traditional class. And yet, both groups achieved essentially the same competence at the end of the class:

    To sum up, OLI was able to teach more students at lower cost and with less time on task from the students yet with the same effectiveness of a traditional class. Another, smaller study with similar traditional vs. cognitive-tutor-hybrid for an introduction to logic course found that while students in the hybrid condition saw a small (though not statistically significant) gap in learning effectiveness relative to their peers, they achieved more than double the completion rate:

    OLI is pretty well known in the broader ed tech community. They show up at conferences. The ITHAKA study got lots of play. I find it odd that we have not had a broader conversation about how they have achieved their results. Because the techniques developed by OLI, their colleagues at LearnLab, and their collaborators in other institutions have serious implications for OER, MOOCs, the textbook industry, and really, the future of education.

    (more…)

  • ITOE: Comparing Two OpenCourseWare Styles

    It’s week three, and the course continues to elide the distinction between open education and open educational resources. That’s a shame because there’s a real opportunity to explore the differences in goals in the current assignment:

    Carefully review five (5) random courses from MIT OCW (http://ocw.mit.edu/) and five (5) random courses from CMU OLI (http://www.cmu.edu/oli/). Write a post on the different ideals of quality expressed in the differing course designs; describe how you feel the different designs reflect on the different universities.

    (more…)