e-Literate

Present is Prologue

Author: Michael Feldstein

  • Personalized Learning Explainer: Teaching to the Back Row

    Phil and I are pleased to share our first animated explainer on personalized learning:

    If it feels like a commercial that could have been produced by a textbook publisher or ed tech vendor, that is entirely intentional. We hope that they will actively promote this video (and the successor that we will be publishing in the next couple of weeks). We are giving them tools they will (hopefully) want to use that move the focus away from product bells and whistles and toward teaching challenges. This attempt at jujitsu is worth some unpacking.

    (more…)

  • Personalized Learning at Law Schools

    I recently had the honor of speaking at the CALI (Center for Computer-Assisted Legal Instruction) conference. I was invited by one of my early heroes, John Mayer. When I first arrived on the ed tech blogging scene, John was already here, doing stuff. He inspired me.

    Anyway, you may or may not know that law schools are currently experiencing an enrollment crisis. As a result, they are accepting students who are below their normal standards. These students are, unsurprisingly, not doing as well (on average) as their predecessors. So I ask the question: Is the problem that the students are “worse,” or is it that nobody is actually teaching law school students, and that the ones being admitted could succeed if only somebody taught them?

    Here’s the video of the keynote, if you’re interested:

  • We’re Giving a Course on Personalized Learning Next Month

    Working with our good friends at ELI, we’re going to be offering a three-session synchronous course called Personalized Learning: Finding the Model That Fits Your Institution July 6th through 20th. As you know, we’re still in early days for personalized learning. Most campus communities are still trying to figure out what it is and what it’s good for—if they’re aware of it at all as a group. Plus, one size most emphatically does not fit all. The goal of the course is to help participants sharpen their own ideas for appropriate ways to facilitate exploration of the topic at their own institutions.

    The format will be highly participatory, especially in the second and third sessions. Phil and I expect that we will all learn a lot from each other.

    Join us!

  • Adaptive Learning Fails to Make the Grade. Or Does It?!

    My latest Chronicle column is up. It analyzes the results of the SRI Education study of the Gates Foundation adaptive learning grantees, some of which we’ve covered in our e-Literate TV case studies. If you’re looking for evidence that adaptive learning is going to deliver on the promise of a robot tutor in the sky, you won’t find it there. But it’s easy to flatten that result into “adaptive learning doesn’t work.” I don’t believe that the SRI study shows any such thing.

    First of all, what is our standard of proof? A good half of my column is devoted to the methodological challenges of doing big meta-studies like this one. It’s really hard to (ethically) control the variables across multiple classrooms well enough to get a clean result. SRI had to throw out most of the data they had for some measures.

    But equally importantly, there’s just a lot that meta-studies can’t get at precisely because the goals of each implementation are different. For example, one of the goals for implementing OLI at UC Davis was getting students more prepared to engage in higher-level critical thinking in class discussion. Here are some Davis faculty talking about their course design goals:

    I’m not sure how one would empirically measure such a result; nor could I see how to incorporate it into a meta-study that also includes, for example, Essex County College’s developmental math course.

    While the Gates Foundation should get credit for bringing in a credible third-party evaluator to review the results of the grants, ((Full disclosure: Our company received grant money from the Gates Foundation to cover personalized learning)) the design parameters for this particular study do not appear to be as useful as they might have been. That said, the larger point is that it’s really hard to do educational research well. Rather than using these studies as Rorchach tests, we should be taking the time to improve our educational research literacy and better understand what each study can and cannot prove.

  • Sakai Is Probably Healthier Than You Think

    These days, most people don’t hear much about Sakai, and when they do, it’s usually bad news. Most recently there was the horrible outage at UC Davis and some other Sakai schools using a support vendor called Scriba. Before that, the last thing you may have heard was the departure of Sakai founders such as Indiana University and University of Michigan for Canvas.

    But there’s more than one way to measure the health of a software project, particularly if it is open source. As it turns out, three of the LMSs with significant US and Canadian market share—Canvas, Moodle, and Sakai—are hosted on Github, where the actual development work is publicly visible. We analyzed the activity for these three projects with the help of our friends at LISTedTECH. What we discovered may surprise you. Sakai development is actually accelerating.

    (more…)

  • SRI’s Study on Gates Personalized Learning Grants Is Out

    This is almost old news now, but we just haven’t been able to dig into it yet. As part of its Adaptive Learning Market Acceleration Program (ALMAP) program, the Gates Foundation funded SRI to do a study of the results of the grants after two years. I hope to finally clear some time to parse through it this week, but two high-level points jump out at me at first glance (neither of which is a big surprise):

    1. There are no conclusive wins here. This is not a robot-tutor-in-the-sky moment. A few programs did well here and there. A handful produced promising incremental gains. But this is not a report that screams, “Wow, adaptive courseware works!” The most you can say is that adaptive learning looks like it could be another arrow in the quiver that helps out in some situations—like developmental math courses at two-year colleges, for example.
    2. Large-scale educational research is incredibly hard and may actually be impossible to do rigorously for certain kinds of questions. I’ll probably get into this more when I’m ready to really write about the report, but one reason the conclusions are murky is because there so many variables in each class—not just each course subject, not just each course at one university, but even with each section of each class taught by one teacher—that really matter, some of which are impossible to control and others of which are unethical to control. It would be a mistake to overinterpret the study as showing that adaptive learning doesn’t help much. I’ve seen studies with narrower focus that have gotten clearer gains. This brings us back to the arrow-in-the-quiver theory. The wider the scope of the research focus, the more that pockets of real benefit will be obscured by noise and uncontrolled variables.

    The report is here.

  • UC Davis, Sakai, and Open Source

    Phil has been a busy boy, putting out two pieces about some Blackboard research that had gotten some negative responses and two more on a horrifically bad LMS outage for UC Davis and other universities using support vendor Scriba.

    Our main schtick here at e-Literate is to get beyond the headlines. We try to explain what is actually happening and why it is important. Often, this involves puncturing hype like “robot tutors in the sky,” not by heaping them with scorn but by showing the gap between the hype and reality. But there are also times when we do the same for anti-hype. For example, there is a popular narrative that Blackboard is evil and bad and stupid, partly based on the massive brand damage the company did to itself a decade ago (and two CEOs ago). As a result, when they do…well…anything, it tends to be interpreted in the worst possible light. Personally, I’m glad that Blackboard is sharing the findings of their product research. I don’t know of many ed tech companies that are doing that. This is different than putting out a white paper about some study the company did showing how awesome their product is. Rather, it shows us how they are trying to understand their customers’ needs. Sharing that back with the world helps us, in turn, understand how Blackboard thinks. It is an important kind of transparency. As Phil’s second post on the topic highlights, we do have to understand that this kind of study is different than an academic study in order to draw the right conclusions from it. But that’s fine.

    Likewise, the Scriba outage story plays into a “Sakai is dying” narrative. As it happens, I was at conference for the foundation that hosts Sakai last week and got something of a rundown on what was going on, including a preview of the release scheduled to come out this summer. (Oddly, nobody mentioned the outage, which must have been in its early stages at that point.) The Sakai sustainability story is complicated enough to merit a separate post, which I will get to later this week. In the meantime, I’d like to put the Scriba outage in context of how open source works in general and how the Sakai ecosystem works in particular. Because it is very easy for this story to reinforce preconceptions rather than looking at it carefully on its own merits.

    Phil wrote,

    There is an interesting angle here in that Sakai is open source yet data is not easily recoverable.

    Right. Open source can help with problems like the Scriba outage. As far as I can tell, it did help UC Davis. But it’s not that simple. There is a lesson to be learned here for all LMS-using schools, but it’s not “open source doesn’t provide the flexibility that it is supposed to provide”; nor is it “Sakai is dying.”

    (more…)