e-Literate

Present is Prologue

Category: Ed Tech

The “Ed Tech” category includes posts about educational technology products themselves, including LMSs and other learning platforms, adaptive learning and other digital curricular materials products, learning analytics, and educational apps of all types. It also includes technical aspects of ed tech products, especially interoperability.

  • Effort and Engagement

    I’ve been thinking a little more this morning about the language used by the researchers in the SJSU Udacity report. They focus a lot on student “effort.” But it’s also pretty common in education to talk about “engagement.” From a technical perspective, the researchers chose the better word. “Effort” is meant to be an observable behavior, e.g., how many minutes students put into watching videos or how many homework problems they solved. “Engagement” is a non-observable attitude that might be a cause for differences in effort that we observe between students. But the connotations of these words tend to encourage different sorts of questions. When we talk about a problem with student effort, we tend to ask how we can get students to do more work. When we talk about a problem with student engagement, we tend to ask how we can get students to want to do more work. The former might lead us to solutions such as student reminders and alerts when they are falling behind or changes in schedule to accommodate students with jobs, while the latter might lead to ideas about increased interactivity or changes to the content.

    Just a thought.

  • What Blackboard, Desire2Learn, and Udacity Should Learn from SJSU

    As Phil noted in his analysisof the SJSU report, one of the main messages of the report seems to be that some of what we already know about performance and critical success factors for more traditional online courses also seem to apply to xMOOCs. But how good is the ed tech industry at taking advantage of what we already know?

    Not very good, as far as I can tell.

    (more…)

  • SJSU research report confirms MOOCs are online courses

    By reading the SJSU research report (download actual report here), one item that really hits me is that however different the scaling model is for MOOCs, they are still online courses and have similar success factors. I am not trying to minimize the value of the report by the title of this post, because there is real value in letting objective data lead you to conclusions, even if hunches and assumptions would have led to similar results.

    First, however, it would be useful to understand the data and key terms – AOLE is Augmented Online Learning Environment, which represents the additional support structures and for-credit status for MOOCs.

    Out of a total of 274 students enrolled in AOLE courses, 249 students remained in the sample after data cleaning, which included removal of students enrolled in multiple courses and of those with no course activity. In addition, 36 students were removed who withdrew from the course or received a final grade of Incomplete. This left 213 students for the deeper analysis. This data set will be referenced throughout this report as a “research data file”.

    So this data does not include no-shows, withdraws or incompletes, making it more akin to analyzing a traditional college course after the add / drop period.

    (more…)

  • SJSU releases NSF-funded research report on Udacity pilot

    Back in late July we found out that San Jose State University was pausing their SJSU Plus pilot program using Udacity for-credit MOOCs due to low passing rates. While there was fairly extensive media coverage of the story broken by Inside Higher Ed, there was the promise of a National Science Foundation (NSF) funded research report based on the spring 2013 courses. I described why the research report was important, despite indications that the lessons learned aligned with well-known practices for online education.

    With this many known issues, which have also been documented in several excellent blog posts, why is it valuable for SJSU to use the external NSF-funded report?

    In my opinion, there will still be value in A) releasing the full data set and analysis, and B) getting others to understand key points.

    Many specialists might understand why the program failed, but many others do not. Campus leaders and policy makers need to learn many of the lessons that are already known in ed tech / higher ed community, and SJSU Plus will help in this area. Plus, I wouldn’t discount some unexpected findings (e.g. need for self-pacing in math courses as mentioned in student surveys). [snip]

    It is important for some of the new participants (and I would add foundations, state governments, university presidents, etc) to fully understand what worked and what didn’t work in this application of online education. The official report from SJSU will provide a valuable service to help with this learning process.

    There is also value in this very public case establishing a precedent of pausing pilots that don’t work, evaluating results early in the process, adjusting the course and support design based on findings, and focusing on student learning outcomes as the primary measure of success.

    After several delays, SJSU has now released the full NSF-funded research report on Elaine Collins’ official web page (she was the principal investigator). The full report can be downloaded here.

    (more…)

  • Some validation of MOOC student patterns graphic

    File this under “you read it first on e-Literate”.

    In previous posts from spring 2013 I provided a graphical view on MOOC student patterns based on observed retention over time as well as differing student types. This graphic was based on anecdotal observations of multiple MOOCs, mostly through Coursera.

    studentPatternsInMoocs3-2

     

    Based on a recent study of the edX Circuits and Electronics MOOC, there is this interesting chart of student patterns based on actual data analysis of the 155k students from the spring 2012 offering of this course. The full report is worth reading, by the way, with some real student pattern insights.

    edX Retention

     

    I took this chart and overlaid it on the MOOC student patterns graphic, scaling for 0% / 100% of enrollment vertically and start / stop of course horizontally.

    Combined MOOC patterns

     

    It’s good to see this validation of the overall retention pattern based on real data analysis to augment the original graphic’s model.

  • What does Devlin Daley’s departure mean for Instructure?

    Despite all the media hype on MOOCs over the past two years, perhaps the most important recent market entry for ed tech has been Canvas, the LMS from Instructure. Instructure was founded in 2008 by Brian Whitmer and Devlin Daley. At the time Brian and Devlin were graduate students at BYU who had just taken a class taught by Josh Coates, where their assignment was to come up with a product and business model to address a specific challenge. Brian and Devlin chose the LMS market based on the poor designs and older architectures dominating the market. This design led to the founding of Instructure, with Josh eventually providing seed funding and becoming CEO by 2010.

    Michael covered the burst of news in January 2011 that served as the launch of Canvas.

    Instructure has just announced that they will be releasing an open source version of their Canvas LMS product. Between this announcement, the winning of the Utah Education Network contract (109,000 college students and 40,000 K12 students), and the oh-so-ever-brief lawsuit by Desire2Learn about that win, Instructure has been making quite a splash lately.

    Since that time Instructure has grown to 7M+ users, 500 customers and 240 employees while raising $50M in total VC funding. With that much early success, it was a surprise to see Devlin Daley leave the company as of last week with his Douglas Adams reference. (more…)

  • SJSU Plus Udacity Pilots: Lack of transparency in describing data

    Alternate Headline: “Our Long National Nightmare is Over – SJSU and Udacity solve problem of college graduates being able to pass remedial math”

    The more I read on SJSU’s announcement on the pilot program, the more troubled I am with the lack of clear description of student population change (I wrote briefly about the change in student populations yesterday). In a nutshell, the spring 2013 pilot was completely different in the major demographic variables than the summer 2013 pilot. That’s good, right, showing that SJSU and Udacity are learning their lessons? It would be good if SJSU clearly described the student differences and avoided any implications that the numbers could be compared. Further, it would be good to avoid misleading comparisons to face-to-face courses at SJSU.

    But that is not what is going on. SJSU, in particular, is going out of its way to compare spring, summer pilots alongside SJSU on-campus courses in its media blitz. And the strategy is working, based on the articles that came directly from SJSU / Udacity interviews and information releases.

    (more…)