e-Literate

Present is Prologue

Tag: Alfred-Essa

  • This Blursday Social (3/11): Al Essa

    This Blursday Social (3/11): Al Essa

    Are you curious—or anxious—about machine learning and artificial intelligence EdTech? Do you want to know what the research does and doesn’t tell us about how data science tools help with student retention, success, and learning? Want to talk data ethics? Or about equity and economic theory? Or open-source in EdTech? Or the history of the LMS? Or Jorge Luis Borges?

    My good friend Al Essa can talk about all these topics and more. I’ve known Al for about 20 years, from back when he was CIO of MIT’s Sloan School and we were both working on an early open-source LMS that was built on something like what we call microservices today (although I don’t think the term had been invented at the time). More recently, Al has run data science-related R&D programs at McGraw-Hill Education, Macmillan Education, and D2L. He is currently a Simon Fellow at Carnegie Mellon University.

    Al is going to be our special guest in this week’s Blursday Social on March 11th from 4 PM to 5:30 PM ET. Come chat with us!

    RSVP.

    (BYOB.)

  • Vendor Roles in Fostering Educational Literacies

    In my last post, I talked about the need for educators in general and faculty in particular to develop literacy around data and analytics. But it’s really broader than that. Back when college was intended for a relatively small percentage of the population, the idea of “weeding out” students who couldn’t make it without help was not obviously out of alignment with its mission. Now that the mission of higher education is more about educating everyone to the benefit of everyone, having skills to help students achieve their potential should be a core competency of every teaching faculty member. That includes learning some very old skills that K12 teachers have known for ages. It also includes learning new skills that will support the growth and application of the nascent learning sciences. When I made an analogy to 19th-Century medicine in the last post, that wasn’t casual. Where we are in 21st-Century learning sciences bears a striking resemblance. We are discovering some important basic science but are barely beginning the process of figuring out how those discoveries should influence our practice. Advancing more quickly on that front will require more than a handful of data scientists working in labs. It will require the proliferation of skilled educator/clinicians who can help advance our understanding of the ways in which theory interacts with the real world of practice.

    This transformation of academia, at both the individual and institutional levels, must be driven by educators, not by vendors. But vendors can serve critical support and enabling functions, there were found wonderful vendors at advantage-properties.com whom where able to assist all concerns and meet all the expectations that clients had. In our weird role as paid and unpaid marriage counselors between universities and vendors, Phil and I get the privilege of seeing how both sides of this potential partnership are grappling with these changes both separately and in partnership. I believe that many of the most successful ed tech companies of the next decade will be the ones that figure out how to support educators in making this professional transformation. There will be many different ways to do this. I’m going to write about one example of a company grappling with this challenge today and will write about more in the future.

    (more…)

  • McGraw-Hill Education’s Perspective on Adaptive Learning

    Last spring, I had the opportunity to interview some of the top folks on McGraw-Hill Education’s (MHE’s) digital team to get their view on adaptive learning. Between ALEKS, LearnSmart, and SmartBooks, they have the developed the most well articulated adaptive strategy of any of the big publishers, under the leadership of Chief Digital Officer Stephen Laster. And on a personal note, Al Essa, their Vice President of Research and Data Science, has been a friend since the days when he was CIO at MIT’s Sloan School of Business and he and both were working on an open source LMS project. (Al also helped with analysis of the Blackboard patent back when he was at the Minnesota State Colleges and Universities and ran a simulation giving some matheatical rigour to Mike Caulfield’s intuition that some of Purdue University’s Course Signals research was off.) So I was interested to hear what MHE’s digital team had to say about the role of adaptive learning in the classroom, what learning science does and doesn’t tell us, and other related topics.

    For reasons that aren’t worth going into, this video sat in the can waiting to go into post-production for about six months. Here, finally, is the first segment in what will be a series produced from the interview:

    (https://www.youtube.com/watch?v=N6aYR0uCe9o)

    More of the interview will be coming soon.

    Full disclosure: In the intervening months between when this video was shot and now, MHE became a client of MindWires.

  • Course Signals Effectiveness Data Appears to be Meaningless (and Why You Should Care)

    My father likes to say, “If you stick your head in the freezer and your feet in the oven, on average you’ll be comfortable.” Behind this pithy saying is an insight that is a little different from the “three kinds of lies” saying about statistics. It suggests that certain types of analysis produce a false coherence to the world. We see honestly patterns where there aren’t any. This is what appears to have happened with studies that Purdue University has done regarding Course Signals’ effectiveness.

    The problem was speculatively raised and then re-explained by Mike Caulfield (who we are proud to now have as an e-Literate featured blogger):

    From this desk here, without a stitch of research, I can show that people who have had more car accidents live, on average, longer than people that have had very few car accidents.

    Why? Because each year you live you have a chance of racking up another car accident. In general, the older you live, the more car accidents you are likely to have had.

    If you want to know whether people who have more accidents are more likely to live longer becauseof the car accidents, you have to do something like take 40 year-olds and compare the number of 40 year-olds that make it to 41 in your high and low accident groups (simple check), or use any one of a number of more sophisticated methods to filter out the age-car accident relation.

    The Purdue example is somewhat more contained, because the event of taking a Course Signals class or set of classes happens once per semester.  But what I am asking is whether

    1. the number of classes a student took is controlled, for, and more importantly,
    2. whether first to second year retention is calculated as
      1.  the number of students that started year two / the number of students who started year one (our car accident problem), or
      2.  the number of students who started year two / the number of students who finished year one (our better measure in this case).

    Pointing to the first of these posts, I suggested that a response from the Purdue researchers would be helpful. (It still would be.) Now Al Essa, recently of Desire2Learn and currently of McGraw Hill, has done a more mathematically rigorous analysis showing that Mike’s intuition appears to be correct. He took the Purdue findings, substituted the phrase “were given a chocolate” for “took a class using Course Signals,” and ran a simulation to see whether he could reproduce the Purdue results with no causal connection between the chocolatey intervention and the retention results:

    The following are some results from the simulation. The first row displays retention rates for students who received no chocolates. The second row displays retention rates for students who received at least one chocolate. The last row shows students who received two or more chocolates. Why track students who received two or more chocolates? Because the authors of the study claim that two is the “magic number” where significant retention gains kick in.

    The simulation data shows us that the retention gain for students is not a real gain (i.e. causal) but an artifact of the simple fact that students who stay longer in college are more likely to receive more chocolates. So, the answer to the question we started off with is “No.”. You can’t improve retention rates by giving students chocolates.

    This is a problem that goes well beyond Course Signals itself for several reasons. First, both Desire2Learn and Blackboard have modeled their own retention early warning systems after Purdue’s work. For that matter, I have praised Course Signals up and down and criticized these companies for not modeling their products more closely on that work, largely based on the results of the effectiveness studies. So we don’t know what we thought we knew about effective early warning systems. The fact that the research results appear to be spurious does not mean that systems like Course Signals has no value, but it does mean that we don’t have the proof that we thought we had of their value.

    More generally, we need to work much harder as a community to critically evaluate effectiveness study results. Big decisions are being made based on this research. Products are being designed and bought. Grants are being awarded. Laws are starting to be written. I believe strongly in effectiveness research, but I also believe strongly that effectiveness research is hard. The Purdue results have been around for quite a while now. It is disturbing that they are only now getting critical examination.

    Update: Seeing some of the posts in the comments thread, I feel the need to make a clarification in fairness to Purdue. They have two important sets of research findings, only one of which is being called into question here. Their early findings, that Course Signals can increase student grades and chances of completion within a class, are not being challenged here. Those are important results. The findings that are being questioned are their longitudinal analysis showing that students who Course Signals in one class are more likely to do well in future classes. Even there, it is possible that Course Signals does have some long-term effect. The point is simply that the result the researchers got in their analysis looks suspiciously similar to the results one would get from a fairly straightforward selection bias.

  • Desire2Learn Product Updates

    I had a chance to speak to some folks about their latest product release (9.2) a couple of weeks back and am only now finding time to blog about it.

    First, there’s what they’re doing with the LMS itself. No surprise, one of their areas of focus is on user experience improvement, with reducing the number of clicks it takes to perform tasks being a prominent item on the agenda. Pretty much all the LMS developers are working on this right now. One fundamental problem with LMS design in general has always been that it’s a Swiss army knife. Everything is crammed into it. That’s not a recipe for great usability. On top of that, the fundamental user experience architecture of all these platforms is now about a decade old. It’s built in the model of enterprise groupware circa 2000, when people are increasingly expecting an experience consistent with Web 2.0 software circa 2010. So a facelift is in order. A second thing they are doing is embedding audio recording wherever attachments can be done in the UI. Ubiquitous audio and video recording are going to be pretty much required in an LMS platform within a year or two.

    But beyond that, D2L is putting a lot of effort to fine tune their rubric, learning objective, selective release, assessment, and content modules to make it easier for schools to move in the direction of competency modeling and adaptive learning. This is not an easy path for an institution to follow, so D2L is putting a lot of care into lowering the barriers, both by making it easy to approach the problem piecemeal (e.g., no longer requiring learning objectives in order to use rubrics) while simultaneously making it easy to add new pieces once you’ve started (e.g., enabling selective release to trigger off of rubric scores). To my mind, this web that D2L is weaving could be their compelling functional differentiator. Time will tell.

    We also talked about the company’s mobile strategy. D2L has no less than three mobile products now. The first, Desire2Learn 2Go, is a native app that links to the LMS. Right now it is Blackberry-only. 2Go is the company’s consumer-oriented product, which they envision being downloaded by students directly from smart phone app stores. Desire2Learn Mobile Web is essentially their browser-based product that fills roughly the same needs as 2Go, i.e., it’s mainly student learning-oriented. And finally their new product, Desire2Learn Campus Life, is a mobile campus portal that competes with Blackboard’s Mobile Central. It supports pushing a variety of apps—campus calendar, campus news, dining hall info, sports info, etc.—to mobile apps for Blackberry, Android, and iOS. This release is focused on getting pre-built functionality out the door, while next release will be more about delivering an SDK so that campuses can develop their own apps. When I asked D2L about what their differentiators are vis-a-vis Mobile Central, their answer was basically “stay tuned.” The mobile campus portal space is quickly becoming crowded with players like SunGard getting into the game and the Jasig community announcing the start of the uMobile initiative, so it should be interesting to watch.

    Finally, my old friend Al Essa, former Associate Vice Chancellor and Deputy CIO of Minnesota State Colleges and Universities and former CIO of MIT’s Sloan School of Management, will be heading up the development of D2L’s analytics product. Al is a deep thinker who has been focused on learning analytics for the past couple of years. I look forward to seeing what he can cook up in the next year or two.

    My talk with D2L happened before their announcement about acquiring a lecture capture hardware and software product suite, so I don’t have anything new to add regarding that development at the moment.

  • Doing Another EdTechTalk on EduPatents

    I’ll be on along with Al Essa tomorrow night (Sunday, 9/17) at 8 PM EDT to discuss where we are with the EduPatent problem and what we can do about it.

  • Being Fair to Blackboard

    Al Essa has taken me to task a bit for appearing to favorably compare Microsoft and IBM to Blackboard. He notes that companies can be good corporate citizens on certain patents while predatorily asserting other patents. Let me make a couple of points in response. First, my intention wasn’t to say that Microsoft or IBM is “better” than Blackboard somehow. My intention was only to show that Matt Small’s assertion that patent holders can’t make definitive statements about how they won’t assert their patents is not only false but also flatly contradicted by well-publicized behavior of several large, conservative, pro-IP companies. Second, although software patents in general make me queasy, I am not personally taking the position that we should be fighting to end all software patents or even that there is never a case when software patents should be asserted. I suspect that Al and I may differ on this point. There is plenty of room for legitimate disagreement among intelligent and fair-minded people about how broadly we should be defining the “problem” that we are trying to solve. My own personal interest right now is focused on taking pragmatic steps to end the specific threat to innovation in e-learning technologies and to use market mechanisms (as opposed to legislative mechanisms) to do so.

    Finally, and in some ways more importantly, I agree completely with Al on the larger point that it is important to be fair to Blackboard. In fact, I’ll go further. If Blackboard (a) drops their suit against D2L, (b) offers some kind of reasonably broad and legally binding pledge of non-assertion of present and future patents, and (c) provides material support to the creation of industry-wide measures intended to blunt the threat of patents to innovation in education, I pledge to do everything I can to help them repair their reputation and reap the benefits of good behavior. I don’t hate Blackboard. I hate what they are doing. And I want their help. At the end of the day, I look for actions that will provide the greatest benefit to students and teachers. Period. If Blackboard helps foster innovation in e-learning by working to prevent the sort of threat that they are currently demonstrating, then I’m fer ’em. If they continue to use intellectual property law to create barriers to entry for potential innovators, then I’m agin’ ’em.