How many times have you heard the statement that 'MOOCs have a completion rate of 10%' or 'MOOCs have a completion rate of less than 10%'? The meme seems to have developed a life of its own, but try to research the original claim and you might find a bunch of circular references or anecdotes of one or two courses. Will the 10% meme hold up once we get more data?
While researching this question for an upcoming post, I found an excellent resource put together by Katy Jordan, a graduate student at The Open University of the UK. In a blog post from Feb 13, 2013, Katy described a new effort of hers to synthesize MOOC completion rate data - from xMOOCs in particular and mostly from Coursera.
Via a combination of thinking about ‘what makes a successful MOOC?’, and looking for a topic for my final project on the Infographics MOOC, I decided to try to pull together the various statistics floating around online about MOOC completion rates. I’m trying to see if any differences emerge on the basis of platform or the assessment methods used.
My draft graph synthesising everything I’ve found so far can be found here: http://www.katyjordan.com/MOOCproject.html
What Katy has done so far is compile completion rate data from 24 MOOCs - 19 from Coursera, 3 from edX, 1 from Udacity and 1 from MITx (a precursor to edX). The data is available in a table format or in a graph.
As you can see on the right side, you can filter by MOOC provider, University or Assessment Type (auto grading, peer grading or some combination).
In the graph view, you can see completion rates vs. total enrollment of each MOOC (the colors represent different Assessment Types).
By clicking on each dot, it highlights the MOOC, basic data and link to data source.
This is the most thorough summary of MOOC completion rate data that I have seen - for xMOOCs. Kudos to Katy for putting together this data, and please comment on her blog post to let her know of new data available.
Some notes based on the data available:
- The average completion rate of xMOOCs is 7.6%, with a minimum of 0.67% and a maximum of 19.2%. The 19.2% appears to be an outlier from Ecole Polytechnique Fédérale de Lausanne, although it may be worth figuring out how they got their rate so high.
- Does it bother anyone that we get this data from a graduate student from The Open University but not from any of the xMOOC providers who claim the power of data analytics in their platforms?
- In the end, the meme of 10% MOOC completion rates is not too far off, at least for Coursera courses.
The problem I see is that scalar metrics of # of students and completion rates are insufficient for open courses, where there are different student types. Only a subset of students actually plan to complete a course, whereas the majority of students enrolled are there only to sample and explore. Measuring completion rate off the total number of students is misleading, but so is touting the total number of students enrolled (billions served) as a key metric. This is not a problem with Katy's data, but with the common discussion of MOOCs in popular media and even higher education circles. I'll address a different model in my next post.
Discuss here or at Google+ post.