e-Literate

Present is Prologue

Tag: MIT

  • MOOCs in Decline: Insights into multi-year data from MIT and Harvard

    MOOCs in Decline: Insights into multi-year data from MIT and Harvard

    It is no secret that the commercial Massive Open Online Courses (MOOCs) launched by Coursera, Udacity, and edX in 2011/12 are a different beast from the majority of for-credit online education offerings. One such difference is the pattern of quick drop-off of students within each course, particularly in the first 2 – 3 weeks, which we documented in 2013.

    We also know that the original disruption vision for these large MOOCs has not come to pass, with many MOOC providers moving into Online Program Management (OPM) business models and shutting off many of the free and open access that made MOOCs in the first place.

    Late last week two researchers at MIT, Justin Reich and José A. Ruipérez-Valiente, published an article in Science titled “The MOOC Pivot” that provides new multi-year insight into the trends based on edX usage at MIT and Harvard, the two founding schools for edX. In a nutshell, MOOC activity peaked in 2016 based on new individual learners, based on a dataset of “565 course iterations from 261 different courses, with a combined 12.67 million course registrations from 5.63 million learners”, leading the MOOC providers to look to the OPM market as a more-realistic method of sustainability. The researchers identify three insights from their data [emphasis added].

    To better understand the reasons for this shift, we highlight three patterns emerging from data on MOOCs provided by Harvard University and Massachusetts Institute of Technology (MIT) via the edX platform: The vast majority of MOOC learners never return after their first year, the growth in MOOC participation has been concentrated almost entirely in the world’s most affluent countries, and the bane of MOOCs—low completion rates—has not improved over 6 years.

    The highlighted conclusion makes sense, but to my knowledge we have previously not had any solid data to back this point up. The following chart in the article shows annual cohorts of learners and tracks them over time, showing that the decrease per year largely mirrors the course-level situation.

    MIT Harvard multi-year research on MOOCs

    Thus 38% of 2012-13 cohort enrolled in at least one course in year 2, and 24% of 2012-13 cohort enrolled in any course in year 3, etc. Over time, these year 2 / year 3 / year 4 retention numbers reduce significantly. Thankfully, Justin and José provided a GitHub repository of the data, and perhaps a transposition of Table S1 can make these trends more obvious.

    It took nearly six years for 10% or fewer of the 2012-13 cohort to remain active (enrolled in any courses), but just two years for 2016-17. And note that the cohort size has been dropping since 2015-16. Keep in mind that the MOOC providers counteract these institution-level trends by increasing the number of institutions that they partner with over time.

    There is more analysis in the full article at Science – it’s worth reading the whole thing.

  • A Moment of Clarity on the Role of Technology in Teaching

    This following excerpt is based on a post first published at The Chronicle of Higher Education.

    With all of the discussion around the role of online education for traditional colleges and universities, over the past month we have seen reminders that key concerns are about people and pedagogy, not technology. And we can thank two elite universities that don’t have large online populations — MIT and George Washington University — for this clarity.

    On April 1, the MIT Online Education Policy Initiative released its report,“Online Education: A Catalyst for Higher Education Reforms.” The Carnegie Corporation-funded group was created in mid-2014, immediately after an earlier initiative looked at the future of online education at MIT. The group’s charter emphasized a broader policy perspective, however, exploring “teaching pedagogy and efficacy, institutional business models, and global educational engagement strategies.”

    While it would be easy to lament that this report comes from a university with few online students and yet dives into how online learning fits in higher education, it would be a mistake to dismiss the report itself. This lack of “in the trenches” experience with for-credit online education helps explain the report’s overemphasis on MOOCs and its underemphasis on access and nontraditional learner support. Still, the MIT group did an excellent job of getting to some critical questions that higher-education institutions need to address. Chief among them is the opportunity to use online tools and approaches to instrument and enable enhanced teaching approaches that aren’t usually possible in traditional classrooms. (more…)

  • Miami, Harvard and MIT: Disability discrimination lawsuits focused on schools as content providers

    In the discussions at Google+ based on last week’s post about the Miami University of Ohio disability discrimination lawsuit ((Insert joke here about G+ and its hundreds of active users.)), George Station made two important points that deserve more visibility.

    It’s been a-coming for several years now. Cal State has some pretty strong rules in place for compliance with ADA and state-level disability laws. Still, [Universal Design for Learning] UDL is a little-known acronym on any campus you care to visit, and staff support is probably one person in an office, except for Miami of Ohio as of this week, I guess…

    Add the recent edX settlement with the US Department of Justice, and the whole direction of edtech changes…

    Put another way, it should come as no surprise that the US Department of Justice is ramping up its enforcement of disability discrimination regulations in the education world. Captioning service provider CaptionSync has an excellent summary of the field, written before the DOJ intervention at Miami. (more…)

  • The Quotable Justin Reich: MOOC research needs to reboot

    Thanks to Audrey Watters I just read a new article in Science Magazine and publicly posted here by Justin Reich, the lead researcher for HarvardX (Harvard’s implementation of edX and associated research team) ((Note that Science Magazine access requires a subscription or purchase or individual article.)). Justin calls out the limitations of current MOOC research that focuses on A/B testing and engagement instead of learning, single-course context, and post hoc analysis with proper course design. While praising the field for making available cleansed data for any type of analysis, his core argument is that we need new approaches that cannot be solved just by research teams.
    Update: Added link to publicly-available DOCX article.

    While the whole article is worth reading, there are quite a few insightful quotes should get past the journal paywall.

    • Big data sets do not, by virtue of their size, inherently possess answers to interesting questions.
    • We have terabytes of data about what students clicked and very little understanding of what changed in their heads.
    • It does not require trillions of event logs to demonstrate that effort is correlated with achievement.

    (more…)

  • No, I don’t believe that Harvard or MIT are hiding edX data

    Since my Sunday post What Harvard and MIT could learn from the University of Phoenix about analytics, there have been a few comments with a common theme about Harvard and MIT perhaps withholding any learner-centered analytics data. As a recap, my argument was:

    Beyond data aggregated over the entire course, the Harvard and MIT edX data provides no insight into learner patterns of behavior over time. Did the discussion forum posts increase or decrease over time, did video access change over time, etc? We don’t know. There is some insight we could obtain by looking at the last transaction event and number of chapters accessed, but the insight would be limited. But learner patterns of behavior can provide real insights, and it is here where the University of Phoenix (UoP) could teach Harvard and MIT some lessons on analytics.

    Some of the comments that are worth addressing:

    “Non-aggregated microdata (or a “person-click” dataset, see http://blogs.edweek.org/edweek/edtechresearcher/2013/06/the_person-click_dataset.html ) are much harder (impossible?) to de-identify. So you are being unfair in comparing this public release of data with internal data analytic efforts.”

    “Agreed. The part I don’t understand is how they still don’t realize how useless this all is. Unless they are collecting better data, but just not sharing it openly, hogging it to themselves until it ‘looks good enough for marketing’ or something.”

    “The edX initiative likely has event-level data to analyze. I don’t blame them for not wanting to share that with the world for free though. That would be a very valuable dataset.”

    The common theme seems to be that there must be learner-centered data over time, but Harvard and MIT chose not to release this data either due to privacy or selfish reasons. This is a valid question to raise, but I see no evidence to back up these suppositions.

    (more…)

  • What Harvard and MIT could learn from the University of Phoenix about analytics

    Last week Harvard and MIT released de-identified data from their edX-based MOOCs. Rather than just produce a summary report, the intent of this release was to open up the data and share it publicly. While it is good to see this approach to Open Data, unfortunately the data set is of limited value, and it actually illustrates a key problem with analytics in higher ed. From MIT News description:

    A research team from Harvard University and MIT has released its third and final promised deliverable — the de-identified learning data — relating to an initial study of online learning based on each institution’s first-year courses on the edX platform.

    Specifically, the dataset contains the original learning data from the 16 HarvardX and MITx courses offered in 2012-13 that formed the basis of the first HarvardX and MITx working papers (released in January) and underpin a suite of powerful open-source interactive visualization tools (released in February).

    At first I was eager to explore the data, but I am not sure how much useful insight is possible due to how the data was collected. The data is structured with one student per row for each course they took (taking multiple courses would lead to multiple rows of data). The data columns (pulled from the Person Course Documentation file) are shown below:

    (more…)

  • Two MOOC curriculum announcements in one week

    In two apparently unrelated announcements, both MIT and Wharton announced they were moving beyond just courses and putting significant parts of their curriculum into MOOC platforms, both with identity verification. MIT is putting several undergraduate sequences online through MITx (their implementation of  edX), while Wharton business school is putting a “foundation series” of first-year courses online through Coursera.

    First the MIT news from Inside Higher Ed (see the Chronicle for additional coverage):

    The Massachusetts Institute of Technology will this fall package some of its online courses into more cohesive sequences, just as edX prepares to roll out certificates of completion using identity verification. Seen together, the two announcements may provide a glimpse at what the future holds for the massive open online course provider.

    The “XSeries” sequences add a new layer of structure to MITx, the institution’s section of the edX platform. The first of seven courses in the Foundations of Computer Science XSeries will be offered this fall, with one or more new courses being rolled out each semester until the fall of 2015. The Supply Chain Management XSeries, consisting of three courses, will begin in the fall of 2014. The two sequences will target undergraduates and working professionals, respectively. [snip]

    (more…)